<meta lang="en" charset="utf-8">

**vk_mini_path_tracer**

A relatively small, beginner-friendly path tracing tutorial.

<small>
By Nia Bickford
</small>

<script type = "preformatted">

This tutorial is a beginner-friendly introduction to writing your own fast,
photorealistic path tracer in less than 300 lines of C++ code and 250 lines of
GLSL shader code using Vulkan. Here's an example of what you'll render at the
end of this tutorial!

![A path traced version of a Cornell box with sky lighting.](images/12-vk_mini_path_tracer.png)

Vulkan is a low-level API for programming GPUs -- fast, highly parallel
processors. It works on a wide variety of platforms -- everything from
workstations, to gaming consoles, to tablets and mobile phones, to edge devices.

Vulkan is usually known as a complex API, but I believe that when presented in
the right way, it's possible to make learning Vulkan accessible to people of all
skill levels, whether they've never programmed graphics before or whether they're a
seasoned rendering engineer. Perhaps surprisingly, one of the best ways to
introduce Vulkan may be with GPU path tracing, because the API involved is
relatively small.

We'll show how to write a small path tracer, using the
NVVK helpers, included in the nvpro-samples framework, to help with some Vulkan calls when needed. For
advanced readers, we'll also optionally talk about performance tips and some of the implementation
details inside the helpers and Vulkan itself. 

The final program uses less than 300 lines of C++ and less than 250 lines of GLSL shader code, including comments. You can find it [here](https://github.com/nvpro-samples/vk_mini_path_tracer/tree/main/vk_mini_path_tracer).

Here are all the Vulkan functions, and NVVK functions and objects, that we'll use in the main tutorial:

Vulkan Functions               |                                |                          
-------------------------------|--------------------------------|-------------------------------
vkAllocateCommandBuffers       | vkBeginCommandBuffer           | vkCmdBindDescriptorSets
vkCmdBindPipeline              | vkCmdDispatch                  | vkCmdFillBuffer
vkCmdPipelineBarrier           | vkCreateCommandPool            | vkCreateComputePipelines
vkDestroyCommandPool           | vkDestroyPipeline              | vkDestroyShaderModule
vkFreeCommandBuffers           | vkGetBufferDeviceAddress       | vkQueueSubmit
vkQueueWaitIdle                | vkUpdateDescriptorSets

NVVK Functions and Objects     |                                  |
-------------------------------|----------------------------------|----------------------------------
nvvk::Buffer                   | NVVK_CHECK                       | nvvk::Context
nvvk::ContextCreateInfo        | nvvk::createShaderModule         | nvvk::DescriptorSetContainer
nvvk::RayTracingBuilderKHR     | nvvk::ResourceAllocatorDedicated |

!!! Tip Link to Code and Chapter Checkpoints on GitHub
    This tutorial's text and code (including chapter checkpoints -- see *Debugging Help* below)
    are available here: https://github.com/nvpro-samples/vk_mini_path_tracer

## Supported GPUs

This sample will work on all GPUs with support for ray queries.
As of this writing, these are all GPUs with hardware-accelerated ray tracing,
including all RTX GPUs. (In particular, GTX GPUs such as 10-series cards don't
support ray queries, but many support [ray tracing pipelines](extras.html#raytracingpipelines). See the box below for more details.)

!!! Tip GPUs Supporting Ray Tracing
    All RTX GPUs have RT Cores, which are dedicated hardware units that make
    acceleration structure traversal and ray-triangle intersections much faster.
    
    Some older GPUs can also emulate kinds of ray tracing a bit more slowly.
    Any GPU listed in the Vulkan Ray Tracing section of the [NVIDIA Vulkan Driver Support][nvbetadriver] page
    supports ray tracing pipelines, which we describe in [the extras](extras.html#raytracingpipelines).
    However, the main section of this tutorial uses Vulkan ray queries,
    which out of NVIDIA GPUs, requires an RTX GPU as of this writing.
    
    Older NVIDIA GPUs (including those that don't support the `VK_KHR_acceleration_structure`
    extension) can also perform ray tracing using [NVIDIA OptiX][optix].
    OptiX 7's API is relatively close to the Vulkan ray tracing pipeline API.
    
    This tutorial uses ray queries because they reduce the amount of C++ code
    involved by about 75 lines, but ray tracing pipelines let shaders call other
    shaders, and can be faster in multi-material scenes. See the box in
    Section [Using Compute Shaders for Ray Tracing] for more information about
    the tradeoffs between ray queries and ray tracing pipelines.
    
    If you have a GPU that supports ray tracing pipelines but not ray queries,
    there are still many ways to learn Vulkan and ray tracing! If you don't have
    prior Vulkan experience, then you can follow this tutorial up to chapter [Acceleration Structures and Ray Tracing],
    skipping section [Requesting Device Extensions],
    then go to the [Ray Tracing Pipelines](extras.html#raytracingpipelines)
    chapter in the extras. If you have some prior Vulkan experience, then check out the
    [NVIDIA Vulkan Ray Tracing Tutorials](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR),
    which will work on your GPU, and go into more detail about ray tracing pipelines
    than this tutorial! Also make sure to check out the
    More Useful Vulkan and Ray Tracing Tutorials section below.

[nvbetadriver]: https://developer.nvidia.com/vulkan-driver
[optix]: https://developer.nvidia.com/optix

## Debugging Help

At the end of each chapter, you'll have a program that you can run and
observe. In addition, we
include versions of the project at the end of each chapter (see the `checkpoints` folder).
You can compute a
file comparison between your project and the reference project, to see if
you've made any typos when copying the code, using a diff tool such as
[Diffchecker](https://www.diffchecker.com/), [Meld](https://meldmerge.org/),
or [WinMerge](https://winmerge.org/).

Finally, if you run into any issues along the way, you can contact the author
at nbickford@nvidia.com or through [GitHub Issues](https://github.com/nvpro-samples/vk_mini_path_tracer/issues).

## More Useful Vulkan and Ray Tracing Tutorials

We wrote this tutorial for programmers of all skill levels. However, there
are many other useful tutorials on Vulkan or on ray tracing available online.
Depending on your goals and your skill level, you may find them helpful
as well.

In particular, if you already have experience with rasterization or Vulkan or
already have a rendering engine that you'd like to use ray tracing for, you
may want to read the [NVIDIA Vulkan Ray Tracing tutorial](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR) by
Martin-Karl Lefrançois, Pascal Gautron, Nia Bickford, and David Akeley.

Here are some other tutorials that might be helpful in combination with this tutorial:

* For a longer Vulkan tutorial about rasterization, [vulkan-tutorial.com](https://vulkan-tutorial.com/) ([Rust version](https://github.com/bwasty/vulkan-tutorial-rs), [Java version](https://github.com/Naitsirc98/Vulkan-Tutorial-Java)).

* For a path tracer tutorial based on Shadertoy (using only GLSL pixel shaders) that goes deeper into materials, Alan Wolfe wrote a path tracer tutorial available at https://blog.demofox.org/2020/05/25/casual-shadertoy-path-tracing-1-basic-camera-diffuse-emissive/.

* For a series of books about ray tracing, the [Ray Tracing in One Weekend](https://github.com/RayTracing/raytracing.github.io) book series (an interesting project is to port these to the GPU for improved performance, such as [a CUDA version you can write](https://developer.nvidia.com/blog/accelerated-ray-tracing-cuda/) and [this Vulkan RTX version](https://github.com/GPSnoopy/RayTracingInVulkan)).

* If you're interested in ray tracing on the GPU with CUDA, you might be interested in OptiX (which has a similar API to Vulkan's ray tracing pipelines). The [OptiX Developer Forum](https://forums.developer.nvidia.com/c/professional-graphics-and-rendering/advanced-graphics/optix/167)'s sticky posts include links to resources, including examples, debugging tools, and profiling tools.

* For more tutorials on many different graphics topics, see the [nvpro-samples](https://github.com/nvpro-samples) collection of samples from NVIDIA.

## Acknowledgements

Thank you to Mathias Heyer and Martin-Karl Lefrançois for reviewing an early version of the code, and to Axel Mamode, Christoph Kubisch, Craig Dean, Daniel Koch, David Akeley, Detlef Roettger, and Tristan Lorach for reviewing the tutorial, finding bugs, and suggesting edits. Thanks to Jeff Bolz, Jeff Juliano, and Tony Zlatinski for helping with synchronization questions. David Akeley researched the information about ray payloads in [the extras](extras.html#raytracingpipelines), with information provided by Eric Werness, Christoph Kubisch, and Daniel Koch. Additional thanks to Alexey Panteleev and Neil Trevett for help during publication.

# Hello, Vulkan!

## Setting Up Your Development Environment

### Install Dependencies

For this tutorial, you'll need to install five things:

* A **C++ compiler** supporting C++14 (on Windows, at least Visual Studio 2017);
* [**Git**][git], to download the sample and library (or download the dependencies as archives from GitHub and follow the instructions [here][git]);
* [**CMake**][cmake], to set up the build system for your operating system (download it [here][cmake]);
* An [**NVIDIA driver**](https://www.nvidia.com/Download/index.aspx) released on or after December 15th, 2020, for the implementations of the new official versions of the Vulkan ray tracing extensions (download it [here](https://www.nvidia.com/Download/index.aspx));
* A [**Vulkan SDK**][vulkansdk] with version at least 1.2.162.0 (released December 15th, 2020), which includes support for the official Khronos (cross-platform) ray tracing extensions (download it [here][vulkansdk]). On Linux, if your distro is compatible with Ubuntu packages, follow the instructions on the [LunarG VulkanSDK Packages](https://packages.lunarg.com/) website; otherwise, please follow the installation instructions on the [*Getting Started with the Linux Tarball Vulkan SDK*](https://vulkan.lunarg.com/doc/sdk/latest/linux/getting_started.html) page.

[git]: https://github.com/nvpro-samples/build_all
[cmake]: https://cmake.org/download/
[vulkansdk]: https://vulkan.lunarg.com/sdk/home

Download and install all these dependencies if your system doesn't have them.
These are available for Windows and Linux operating systems.

## Download the Project

**The preferred way** to download the project (including NVVK) is to use the
nvpro-samples `build_all` script.

In a command line, clone the `nvpro-samples/build_all` repository from
[https://github.com/nvpro-samples/build_all](https://github.com/nvpro-samples/build_all):

~~~~~
git clone https://github.com/nvpro-samples/build_all.git
~~~~~

Then open the `build_all` folder and run either `clone_all.bat` (Windows) or
`clone_all.sh` (Linux).

**If you want to clone as few repositories as possible**, open a command line,
and run the following commands to clone the repositories you need:
~~~~~
git clone https://github.com/nvpro-samples/nvpro_core.git
git clone https://github.com/nvpro-samples/vk_mini_path_tracer.git
~~~~~

## Configure and Generate the Project using CMake
Launch the CMake GUI. In the "Where is the source code" box, enter the path to
`vk_mini_path_tracer`. Then in "Where to build the binaries", enter a path to
a temporary build folder, such as `vk_mini_path_tracer`​`/cmake_build`.

![](images/1-cmakeDirectories.png)

Click on "Configure", and set your generator. Most of the time, the default
is fine. Make sure to use the `x64` platform for the generator!

![](images/1-cmakeGenerator.png)

Click on "Finish". Then once configuration finishes, click on "Generate", and
then "Open Project". Your IDE should open. Here's what this looks like in
Visual Studio:

![](images/1-vsProjects.png)

Once you're inside Visual Studio, make sure to right-click on `vk_mini_path_tracer__edit` in the Solution Explorer and select `⚙ Set as Startup Project`. You'll also want to switch to the Debug configuration. See below for more details.

!!! Tip Use Debug Builds for Development
    If you're using Visual Studio, make sure to use the "Debug" solution configuration.
    
    If you're using a single-configuration generator (e.g. the default Makefiles or Ninja
    generators), you'll need to set `CMAKE_BUILD_TYPE` to `Debug` inside CMake. Here's how
    to do that:
    
    - If you're using the CMake command line (instead of the GUI), you can do this by adding `-DCMAKE_BUILD_TYPE=Debug` to the `cmake` command line.
    
    - Otherwise, if you're using the CMake GUI, then before configuration, you can click on "Add Entry".
    
    ![](images/1-cmakeDebugAdd.png)
    
     - Then set "Name" to `CMAKE_BUILD_TYPE`, "Type" to `STRING`, "Value" to `Debug`, and then click "OK". Then configure the project as usual.
    
    ![](images/1-cmakeDebugGUI.png)
    
    This and several later chapters make use of the **validation layer**'s messages, which (as of this writing) are only shown in Debug builds. GPU performance will be about the same as in Release builds.

!!! Tip Setting the Startup Project in Visual Studio
    If you're using Visual Studio, the current startup project will be shown in **bold** in
    the Solution Explorer.  For example, in the above image, `ALL_BUILD` is in bold as it is
    the first project in the solution.

    If you try to start debugging by pressing `F5`, or pressing the `▶ Local Windows Debugger` button then you
    are likely to see an error dialog stating something similar to `Unable to start program '<YOUR PATH>\ALL_BUILD'.  Access is denied.`

    To address this issue, right click on the `vk_mini_path_tracer__edit` project (which is the one
    you will be using to follow along with this tutorial) in the Solution Explorer and select `⚙ Set as Startup Project`.

You're now ready to begin programming with Vulkan!

## Hello, Vulkan!

Open `main.cpp` in the `vk_mini_path_tracer_edit` project. This is the project
that you'll edit throughout this tutorial. `Main.cpp` should currently look like
this, a Hello World sample:

~~~~ cpp linenumbers
#include <iostream>

int main(int argc, const char** argv)
{
  std::cout << "Hello world!";
}
~~~~

Make sure that this compiles and runs.

Now, edit `main.cpp` so that it looks like this -- we'll explain what this does
after the code:

~~~~ cpp linenumbers
#include <nvvk/context_vk.hpp>

int main(int argc, const char** argv)
{
  // Create the Vulkan context, consisting of an instance, device, physical device, and queues.
  nvvk::ContextCreateInfo deviceInfo;  // One can modify this to load different extensions or pick the Vulkan core version
  nvvk::Context context;               // Encapsulates device state in a single object
  context.init(deviceInfo);            // Initialize the context
  context.deinit();                    // Don't forget to clean up at the end of the program!
}
~~~~

This is a fully functioning program that uses the Vulkan API! You can even run
it, and it will output some interesting information about your GPU and system.

First, we create an `nvvk::ContextCreateInfo` object. We'll use this in
subsequent samples to tell `nvvk::Context::init` to request different Vulkan features.
Now, we don't load anything extra.

Then we initialize an `nvvk::Context` using these settings. We'll go into more
detail what an `nvvk::Context` is shortly, but for the moment, you can think of
it as representing a single GPU.

Finally, we deinitialize the context. It's usually a good idea to clean up the
resources you've allocated at the end of a program.

Here's an example of the output of this program; depending on your GPU and
system, you may see different things listed. By looking at this output, we can
get a better idea of what an `nvvk::Context` is.

~~~~ none
_______________
Vulkan Version:
 - available:  1.2.154
 - requesting: 1.1.0
___________________________
Available Instance Layers :
VK_LAYER_NV_optimus (v. 40209b 1) : NVIDIA Optimus layer
(more VK_LAYER names here...)

Available Instance Extensions :
VK_KHR_device_group_creation (v. 1)
VK_KHR_external_fence_capabilities (v. 1)
(more instance extensions here...)
______________________
Used Instance Layers :
VK_LAYER_KHRONOS_validation

Used Instance Extensions :
VK_EXT_debug_utils
____________________
Compatible Devices :
0: GeForce RTX 2070
Physical devices found : 1
_____________________________
Available Device Extensions :
VK_KHR_16bit_storage (v. 1)
VK_KHR_8bit_storage (v. 1)
VK_KHR_bind_memory2 (v. 1)
(many more device extensions here...)
________________________
Used Device Extensions :

INFO: Loader Message
 --> Inserted device layer VK_LAYER_KHRONOS_validation (C:\VulkanSDK\1.2.148.1\Bin\.\VkLayer_khronos_validation.dll)
INFO: Loader Message
 --> Inserted device layer VK_LAYER_NV_optimus (C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_4f9409a167e6e873\.\nvoglv64.dll)

~~~~

Now, let's take a look at what the information above means, and what it means
that an `nvvk::Context` includes an instance, a device, a physical device, and queues.

## `nvvk::Context`: The Vulkan Instance

In some sense, you can think of Vulkan as a way to use a CPU to program GPUs.
When the CPU sends a command to the GPU via Vulkan, there are many systems that
the command must pass through to result in an operation on the GPU.

************************************************
*
*   .-------------.
*   | Application |
*   '-------------'
*          ^
*          |
*          v
*  .---------------.
*  | Vulkan shared |
*  |    library    |
*  '---------------'
*          ^
*          |
*          v
*    .------------.                  .-----.
*    | GPU driver | <---PCIe bus---> | GPU |
*    '------------'                  '-----'
*
************************************************

One initial abstraction is as follows: An application talks to the Vulkan
library, which then talks to the GPU's driver, which sends commands and data over
a PCIe bus (for instance) to the GPU.

When you install a modern GPU driver, you also install a version of the Vulkan
shared library. (Indeed, on Windows, there should be a `vulkan-1.dll` file in
`C:\Windows\System32`.)

So, when you initialize a **Vulkan instance**, it loads and initializes this
Vulkan library.

### Instance Extensions and Layers

One of the most important things about Vulkan is that you can enable additional
functionality and features using layers and extensions. **Vulkan extensions**
correspond to different things your GPU can do beyond core functionality, while information passes through
different **Vulkan layers** before it reaches your GPU driver.

For example, you'll notice that the application already uses the
`VK_LAYER_KHRONOS_validation` instance layer (listed under `Used Instance Layers`).
This is a helpful layer
for debugging: the validation layer keeps track of the current state of all
your objects, and whenever you call a Vulkan function, this layer updates its
state, and lets one know if anything they've done could violate the Vulkan
specification (e.g. if it performs an invalid Vulkan call). It also includes object
information, so they can determine what went wrong.

To make debugging easier, one can also add the `VK_EXT_debug_utils` extension,
which allows one to give names to the Vulkan objects they create. These debug
names will then appear in the validation layer's messages, which can help in
determining which objects are involved. We describe how to do this in [the extras](extras.html#debugnames).

!!! Tip
    Make sure you're using a debug build as described in Section [Configure and Generate the Project using CMake] in order to see validation layer messages.

Let's show an example of a validation layer error message.

The Vulkan function [`vkAllocateCommandBuffers`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkAllocateCommandBuffers.html) has the following signature:

~~~ cpp
VkResult vkAllocateCommandBuffers(
    VkDevice                                    device,
    const VkCommandBufferAllocateInfo*          pAllocateInfo,
    VkCommandBuffer*                            pCommandBuffers);
~~~

The Vulkan specification for [`vkAllocateCommandBuffers`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkAllocateCommandBuffers.html) also states:

> **Valid Usage (Implicit)**
>
> * `device` **must** be a valid [VkDevice](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkDevice.html) handle
> * `pAllocateInfo` **must** be a valid pointer to a [VkCommandBufferAllocateInfo](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkCommandBufferAllocateInfo.html) structure
> * `pCommandBuffers` **must** be a valid pointer to an array of `pAllocateInfo->commandBufferCount` [VkCommandBuffer](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkCommandBuffer.html) handles
> * `pAllocateInfo->commandBufferCount` **must** be greater than `0`

Let's try to cause an error by calling `vkAllocateCommandBuffers` with *invalid*
`pAllocateInfo` and `pCommandBuffers` pointers, such as `nullptr`:

~~~~ cpp
#include <nvvk/context_vk.hpp>

int main(int argc, const char** argv)
{
  // Create the Vulkan context, consisting of an instance, device, physical device, and queues.
  nvvk::ContextCreateInfo deviceInfo;  // One can modify this to load different extensions
  nvvk::Context           context;     // Encapsulates device state in a single object
  context.init(deviceInfo);            // Initialize the context
~~~ cpp highlight
  vkAllocateCommandBuffers(context, nullptr, nullptr); // Invalid call!
~~~ cpp
  context.deinit();                    // Don't forget to clean up at the end of the program!
}
~~~~

!!! Tip
    We haven't described [command buffers](index.html#commands) yet, but you can ignore them in this case.

If you run this, the Vulkan validation layer will warn you that `pAllocateInfo` isn't a valid `VkCommandBufferAllocateInfo` pointer:

~~~~ output
ERROR: VUID-vkAllocateCommandBuffers-pAllocateInfo-parameter
 --> Validation Error: [ VUID-vkAllocateCommandBuffers-pAllocateInfo-parameter ]
 Object 0: handle = 0x158dfd26f68, type = VK_OBJECT_TYPE_DEVICE; |
 MessageID = 0x72e32441 | vkAllocateCommandBuffers: required parameter
 pAllocateInfo specified as NULL The Vulkan spec states: pAllocateInfo must be a
 valid pointer to a valid VkCommandBufferAllocateInfo structure
 (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-vkAllocateCommandBuffers-pAllocateInfo-parameter)
~~~~

If you added it, make sure to remove the incorrect
`vkAllocateCommandBuffers` call before continuing.

!!! Tip
    The application enabled the validation layer because `bUseValidation` is
    true by default in `nvvk::ContextCreateInfo`'s constructor:
    ~~~~ cpp
    ContextCreateInfo(bool bUseValidation = true);
    ~~~~

As opposed to the longer list of device extensions, **instance extensions** are
Vulkan extensions that apply to the application's entire Vulkan instance,
instead of for a single GPU. For instance, we only have one instance extension
enabled now, `VK_EXT_debug_utils`; this allows the entire application
to log debug messages (such as the validation error above).

!!! Tip Vulkan Extension Support
    [Vulkan.gpuinfo.org](http://vulkan.gpuinfo.org/listextensions.php) is a useful resource for
    finding out which GPUs support which extensions, including more than 1000 desktop and
    mobile GPUs across generations. It also includes information about many other Vulkan properties.

## `nvvk::Context`: The Physical Device and the (Logical) Device

After the instance extensions, the output lists all the GPUs on your system
that support Vulkan. For instance, when running on a laptop with an integrated
and a discrete GPU, this will list two **physical devices**:

~~~~ output
Compatible Devices :
0: GeForce GTX 1060 with Max-Q Design
1: Intel(R) HD Graphics 630
Physical devices found : 2
~~~~

**********************************
*
*        .-------------.
*        | Application |
*        '-------------'
*               ^
*               |
*               v
*          .--------.
*          | Vulkan |
*          '--------'
*           ^      ^
*           |      |
*           v      v
* .------------. .------------.
* | Discrete   | | Integrated |
* | GPU Driver | | GPU Driver |
* '------------' '------------'
*       ^              ^
*       |              |
*       v              v
*  .----------.  .------------.
*  | Discrete |  | Integrated |
*  | GPU (0)  |  | GPU (1)    |
*  '----------'  '------------'
*
**********************************

Generally, each physical device corresponds to a physical GPU, with memory,
processing units, connections to other devices, and the paths in silicon
to implement different features. Each physical device may also sometimes involve
a different driver -- for instance, the NVIDIA GTX 1060 Max-Q GPU uses a
different driver than the Intel integrated GPU.

A **logical device**, or a **device** for short, usually points to a physical
device. When `nvvk::Context` creates the device, it also specifies a set of
**device extensions**, and Vulkan checks to see if the physical device supports
these device extensions. One can think of a device as an environment usually used for
communicating with a physical device. Vulkan uses `VkDevice` handles for devices, and
`VkPhysicalDevice` handles for physical devices.

!!! Tip Multi-GPU Programming
    Some applications like [Quake 2 RTX](https://github.com/NVIDIA/Q2RTX) can use multiple GPUs at once!
    In addition, if one links together GPUs using SLI or NVLink, then they can
    create a logical device that points to two or more physical devices.

The most important device extensions for this sample are `VK_KHR_acceleration_structure` and `VK_KHR_ray_query`,
which the application can enable if the physical device supports ray tracing.
Once we enable that, we can call the Vulkan ray tracing functions!

!!! Tip
    The validation layer will usually let you know if using a function requires
    an extension that the application hasn't enabled. The Vulkan specification
    also documents device extensions.

## `nvvk::Context`: Queues

Many GPUs support multiple **queues**. We won't describe them in detail here,
since we only use one queue for this sample.

In Vulkan, you submit work to queues. (This work will start in order, but might complete out
of order -- we'll talk about this more in the Synchronization sections.)
Different queue families support different types of operations.
For instance, the graphics queue family supports graphics
operations (e.g. ray tracing), while the transfer queue family supports data transfer
operations (e.g. clearing and copying data). We'll only use one queue in one
queue family -- the graphics, compute, and transfer queue family, which supports
all types of Vulkan operations (graphics operations, compute operations, and transfer operations).

## Summary

In this chapter, we saw how to initialize a Vulkan instance, create a Vulkan
physical device and logical device, and set up queues, by initializing an
`nvvk::Context`. We also described several different Vulkan objects:

* A **Vulkan instance** (`VkInstance`) represents an instance of the Vulkan library.
* A **Vulkan device** (`VkDevice`) points to one or multiple physical devices (`VkPhysicalDevice`s), which correspond to GPUs; it is our primary object used to communicate with the GPU.
* Information passes through **Vulkan instance layers** before reaching the driver.
* An application can enable different **Vulkan extensions** to access features such as ray tracing.






<!-- Chapter 2 -->
# Device Extensions and Vulkan Objects

In this chapter, we'll show how to:
* Specify a version of the Vulkan API for instance creation;
* Request device extensions including the `VK_KHR_acceleration_structure` and `VK_KHR_ray_query` extensions;
* Find out information about a GPU by getting Vulkan features and physical device properties;
* Create lists of Vulkan objects using `pNext` chains.

!!! Tip Troubleshooting
    If you run into any issues during this chapter, you can compare your code
    against the reference code at the end of this chapter provided in the `checkpoints/2_extensions` folder. For instance,
    you can use a diff tool such as [Meld](https://meldmerge.org/) or [Diffchecker](https://www.diffchecker.com/) to automatically find any differences between
    your code and the reference code.
        
    If your code reports that no device supports the extensions in this chapter,
    make sure you have the latest driver installed as described in Section [Install Dependencies],
    and that your system has a ray tracing-capable GPU such as an RTX GPU. You can also contact the author at nbickford@nvidia.com, or create a [GitHub issue](https://github.com/nvpro-samples/vk_mini_path_tracer/issues).

## Setting a Vulkan API Version

Like most APIs, Vulkan is expected to slowly change over the years, adding new
features and sometimes deprecating older features.
To handle these changes, Vulkan uses different version numbers,
device extensions, and extensible structures.

You might have noticed that the output of last chapter's project said that
we were only requesting Vulkan version `1.1.0`. As of this writing, the current
`major.minor` version of Vulkan is `1.3`. Each new `major.minor` version of Vulkan
includes a set of extensions by default, meaning that we don't have to request
them if we successfully request the version.

We can request Vulkan version 1.2 as follows:
After creating the `nvvk::ContextCreateInfo` object but before calling `context.init`, add the following lines:

~~~ cpp
  deviceInfo.apiMajor = 1;  // Specify the version of Vulkan we'll use
  deviceInfo.apiMinor = 2;
~~~

If you run the project, you should now see that the output starts with something
like this:

~~~ output
_______________
Vulkan Version:
 - available:  1.2.154
 - requesting: 1.2.0
~~~

## Three Ways to Initialize Vulkan Structures

Vulkan structures often include `sType` members, which the application needs
to set to the structure's [`VkStructureType`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkStructureType.html).

!!! Tip Why do we need to set `sType`?
    Vulkan often refers to objects using `void` pointers. To determine
    the type of a Vulkan object that a `void *` points to, Vulkan looks at the
    first 32 bits of the structure, which contain `sType`, the structure type number.

Initializing Vulkan structures can be tedious, especially if the variable name
repeats over several lines. Here's an example:

~~~ cpp
VkCommandBufferAllocateInfo cmdAllocInfo;
cmdAllocInfo.sType              = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
cmdAllocInfo.commandPool        = cmdPool;
cmdAllocInfo.level              = VK_COMMAND_BUFFER_LEVEL_PRIMARU;
cmdAllocInfo.commandBufferCount = 1;
~~~

There are several ways we can make this shorter:

### Using Uniform Initialization

Combine a declaration and set its `sType` in a single line, like this:

~~~ cpp
VkCommandBufferAllocateInfo rtFeatures{VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO};
~~~

The rest of the struct members are automatically set to 0.

We can also use uniform initialization to set multiple struct members at once,
by listing the value of each struct member in order. But since uniform
initialization doesn't list the member names, it can be a bit confusing to read
unless one has the definition of the struct to compare against:

~~~ cpp
VkCommandBufferAllocateInfo rtFeatures{VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO, nullptr, cmdPool,
                                       VK_COMMAND_BUFFER_LEVEL_PRIMARY, 1};
~~~

### Using Designated Initialization

In C99, C++20, and later language versions, you can initialize a struct by
listing each member you want to set and its value. Members you don't list are
set to 0 -- so, for example, this code sets `cmdAllocInfo.sType` and sets the
rest of the struct's members to 0:

~~~ cpp
VkCommandBufferAllocateInfo rtFeatures{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO};
~~~

And this code sets multiple struct members:

~~~ cpp
VkCommandBufferAllocateInfo cmdAllocInfo{.sType              = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,
                                         .commandPool        = cmdPool,
                                         .level              = VK_COMMAND_BUFFER_LEVEL_PRIMARY,
                                         .commandBufferCount = 1};
~~~

Notice that since designated initialization lists the member names, it's clearer
now which members are set to which values!

Designated initializers can even be used to initialize nested structures! So we
could initialize a struct that appears later on like this:

~~~ cpp
VkAccelerationStructureGeometryTrianglesDataKHR triangles{
    .sType         = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_GEOMETRY_TRIANGLES_DATA_KHR,
    .vertexFormat  = VK_FORMAT_R32G32B32_SFLOAT,
    .vertexData    = {.deviceAddress = vertexBufferAddress},
    ...
    .indexData     = {.deviceAddress = indexBufferAddress},
    .transformData = {.deviceAddress = 0}
};
~~~

For the rest of the tutorial, we'll use a combination of uniform and designated
initialization.

### Using Vulkan-Hpp

The Vulkan SDK also comes with [Vulkan-Hpp](https://github.com/KhronosGroup/Vulkan-Hpp), a set of C++ bindings for the Vulkan
API that include additional type safety and more C++ features. When initialized,
Vulkan-Hpp objects automatically set their `sType` fields.

When using all Vulkan-Hpp's features, the final C++ code for this tutorial
becomes about 50 lines shorter, from about 300 to about 250 lines of code.

However, since the C-based Vulkan API is more widely used and some readers may
not be familiar with the aspects of C++ the Vulkan-Hpp library uses, this
tutorial uses the Vulkan C API.

If you're interested in Vulkan-Hpp, we encourage you to check out Andreas Süßenbach's
introduction to it at https://developer.nvidia.com/blog/preferring-compile-time-errors-over-runtime-errors-with-vulkan-hpp.

## Requesting Device Extensions

There are three Vulkan KHR ray tracing extensions:

- [`VK_KHR_acceleration_structure`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_KHR_acceleration_structure.html) enables support for **acceleration structures**. These are data structures that can store geometry (like models) in a way that allows them to be quickly ray cast.

- [`VK_KHR_ray_query`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_KHR_ray_query.html) enables **compute pipelines** and **graphics pipelines** to cast rays.

- [`VK_KHR_ray_tracing_pipeline`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_KHR_ray_tracing_pipeline.html) adds **ray tracing pipelines**, which lets code work with tables of shaders (GPU code), as well as shaders that can call other shaders. This can be more efficient than ray queries in scenes with many different materials.

This main page of the tutorial will use the `VK_KHR_ray_query` extension, because it's slightly faster to get started with. We'll show how to use `VK_KHR_ray_tracing_pipeline` in [the extras](extras.html#raytracingpipelines).

<!-- Align this paragraph to the left on account of non-breaking code blocks -->
<p style="text-align:left">
If you look at the specification for `VK_KHR_ray_query`, you'll notice that it requires `VK_KHR_acceleration_structure`.
This extension, in turn, requires `VK_KHR_deferred_host_operations`.
All of the other required extensions are included in Vulkan 1.2 (which we requested in Section [Setting a Vulkan API Version]),
so we only need to request these three extensions: `VK_KHR_ray_query`, `VK_KHR_acceleration_structure`, and `VK_KHR_deferred_host_operations`.
</p>

**************************************
*
*         .------------------.
*         | VK_KHR_ray_query |
*         '--------+---------'
*                  |
*                  v
*  .-------------------------------.
*  | VK_KHR_acceleration_structure |
*  '---------------+---------------'
*                  |
*                  v
* .---------------------------------.
* | VK_KHR_deferred_host_operations |
* '---------------------------------'
*
**************************************
[Dependency graph of the main tutorial's Vulkan extensions. Extensions included in Vulkan 1.2 not pictured. An arrow from A to B indicates that extension A depends on extension B.]

!!! Tip What is `VK_KHR_deferred_host_operations` for?
    We won't use `VK_KHR_deferred_host_operations` directly, but some readers may be
    interested in knowing about what it does.
    
    `VK_KHR_deferred_host_operations` allows the driver to run some expensive
    CPU-based Vulkan API calls asynchronously (such as a Vulkan API call that
    builds an acceleration structure on a CPU instead of a GPU) -- much like
    launching a thread in C++ to perform a task asynchronously, then waiting for
    it to complete. Note that unlike OpenGL, one can multithread Vulkan calls 
    (although one must ensure that two threads don't use the same resource
    in calls that require that resource to be externally synchronized) --
    and using this extension isn't needed to do multithreading.

For `VK_KHR_deferred_host_operations`, we only need to add it as a required
device extension using `nvvk::ContextCreateInfo::addDeviceExtension`.
After setting `deviceInfo.apiMinor`, add the following code:

~~~ cpp
  // Required by VK_KHR_ray_query; allows work to be offloaded onto background threads and parallelized
  deviceInfo.addDeviceExtension(VK_KHR_DEFERRED_HOST_OPERATIONS_EXTENSION_NAME);
~~~

When we request the `VK_KHR_ray_query` and `VK_KHR_acceleration_structure` extensions, we can
also get structures listing which features of these extensions they support. In particular, we'll make
sure that the device supports both the ray query feature and the acceleration structure feature.

<!-- Align this paragraph to the left on account of non-breaking code blocks -->
<p style="text-align:left">
Let's consider `VK_KHR_acceleration_structure` first. To do this for this extension,
we'll create an empty `VkPhysicalDeviceAccelerationStructureFeaturesKHR` object,
then include a pointer to this object in our `addDeviceExtension` call. When we
initialize the `nvvk::Context`, Vulkan will determine which acceleration structure
features the GPU supports, and tell us which ones it supports through
`VkPhysicalDeviceAccelerationStructureFeaturesKHR`'s fields.
</p>

Add the following code after the last `addDeviceExtension` call:

~~~ cpp
  VkPhysicalDeviceAccelerationStructureFeaturesKHR asFeatures{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ACCELERATION_STRUCTURE_FEATURES_KHR};
  deviceInfo.addDeviceExtension(VK_KHR_ACCELERATION_STRUCTURE_EXTENSION_NAME, false, &asFeatures);
~~~

!!! Tip
    The second argument here, `false`, tells `deviceInfo` that the
    `VK_KHR_acceleration_structure` feature is not optional (that is, it is required). `nvvk::Context::init`
    will throw an assertion error if it couldn't find any devices that supported
    all required extensions.

<!-- Align this paragraph to the left on account of non-breaking code blocks -->
<p style="text-align:left">
`VK_KHR_ray_query` works the same way, where we include a pointer to a `VkPhysicalDeviceRayQueryFeaturesKHR` object:
</p>

~~~ cpp
  VkPhysicalDeviceRayQueryFeaturesKHR rayQueryFeatures{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_RAY_QUERY_FEATURES_KHR};
  deviceInfo.addDeviceExtension(VK_KHR_RAY_QUERY_EXTENSION_NAME, false, &rayQueryFeatures);
~~~

Now, run the program. If all goes well, you'll notice that the output now lists
all three device extensions:

~~~ output
________________________
Used Device Extensions :
VK_KHR_deferred_host_operations
VK_KHR_acceleration_structure
VK_KHR_ray_query
~~~

If you add a breakpoint after `context.init`, you can also examine `asFeatures` or `rayQueryFeatures`
to determine what acceleration structure or ray query functionality your GPU and driver supports!

![This device supports acceleration structures, replaying captures of
acceleration structures, and updating acceleration structure descriptors after
binding a descriptor set (see Section [Descriptors]).](images/2-rtFeatures.png)

Finally, if you run this application on a system where only one GPU supports
acceleration structures and ray queries, you'll also notice that the list of compatible devices changed.


<!-- Chapter -->

# Memory

This chapter is about GPU memory, and how to read GPU memory from the CPU.

In this chapter, we'll:

* Initialize an NVVK GPU memory allocator;
* Allocate a `VkBuffer` to hold color data on the GPU;
* Map the (uninitialized) `VkBuffer` data back to the CPU and print out the first few values.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/3_memory` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Data Transfer Overview

In many ways, a discrete GPU works like a second computer that is extremely good
at parallel data processing, with its own memory and thousands of cores.

However, data transfer between the CPU and GPU is slower than accessing video
RAM (VRAM) from the GPU or accessing RAM from the CPU, and is limited by the
speed of the (usually PCIe) connections between the motherboard and the GPU.
CPU-GPU transfers are also much higher-latency than memory writes and reads.

******************************************
*
*  .-----.                  .----------.
*  | RAM | <-- ~32 GB/s --> |   VRAM   |
*  '-----'                  '----------'
*     ^                           ^
*     |                           |
*     |                           |
* ~102 GB/s                   ~936 GB/s
*     |                           |
*     |                           |
*     v                           v
*  .-----.                  .-----------.
*  | CPU |                  | GPU Cores |
*  '-----'                  '-----------'
*
******************************************
[Diagram of approximate data transfer speeds between the CPU, RAM, VRAM, and GPU cores with recent technology. All figures are theoretical limits and affected by things like usage and coding. GPU-VRAM bandwidth from [a 384-bit bus with GDDR6X memory at a speed of 19.5 gigatransactions/s](https://wccftech.com/nvidia-geforce-rtx-3090-24-gb-official-launch-price-specs-performance). One-way PCIe bandwidth from [a PCIe 4.0 bus with 16 gigatransactions/(lane*s) and 16 lanes](http://rastergrid.com/blog/gpu-tech/2020/11/memory-types-of-discrete-gpus/). RAM bandwidth from [a DDR5 module with 6.4 gigatransactions/s and 128 bits per transaction](https://www.extremetech.com/computing/312730-ddr5-memory-specification-finalized-up-to-6400gt-s-2tb-lrdimms).]

As a result, it's a good idea to minimize the amount of data transferred between
the CPU and GPU, and to minimize synchronization between the two (so that this
latency doesn't become an issue, and so that the CPU and GPU can work at the
same time as often as possible). For instance, we'll upload geometry data to the GPU, do as much
computation as possible entirely on the GPU, and then only copy the image data
back to the CPU at the end. Applications might also send compressed data to the
GPU, which the GPU could either decompress or read without decompressing.

This is also a major factor in the design of Vulkan. Instead of potentially
sending data between the CPU and GPU with every command, Vulkan's design
encourages applications to batch collections of operations in **command buffers**,
then submit these larger packets of information to the GPU all at once.

!!! Tip Future GPU Upload Techniques
    There are more ways to send data to the GPU than by using the CPU. For
    instance, the [NVIDIA GPUDirect API](https://developer.nvidia.com/gpudirect)
    allows devices like SSDs and network adapters
    to read and write GPU memory without going through the CPU, and the upcoming
    (as of this writing) DirectStorage API will add NVMe-GPU transfers to DirectX.

!!! Tip Advanced: Interesting Command Buffer Features
    Vulkan is designed for multithreaded applications; a multithreaded application
    can record command buffers on multiple threads, for instance (although
    it still needs to use synchronization techniques like mutexes to avoid resource
    write hazards, like two threads modifying device state at once).
    
    It's also possible to record **secondary command buffers**, and then re-use
    these secondary command buffers in multiple places in primary command buffers
    (the type of command buffer that this tutorial works with).
    
    For more information about command buffers, please see Christoph Kubisch's
    article [Engaging the Voyage to Vulkan](https://developer.nvidia.com/engaging-voyage-vulkan).

## Including Additional Headers

Add the following line to the list of `#include` directives at the top of
`main.cpp`. This will include the NVVK memory allocator that we'll use.

~~~ cpp
#include <nvvk/resourceallocator_vk.hpp>  // For NVVK memory allocators
~~~

## Memory Allocation in Vulkan

Memory allocation in Vulkan might be a bit lower-level than you're used to,
but it makes sense when you think about how complex memory allocation can be.

When programming on the CPU using a language such as C++, you might know that
memory allocation functions such as `new` and `malloc` are more complex than
they appear. For instance, a function such as `malloc` might use lower-level
functions to allocate relatively
large regions of heap memory from the OS, then suballocate the requested amount
of memory from this region. It also must keep track of which sections of memory
the application allocated, as well as the size of each suballocation.
(Dan Luu describes an example implementation of `malloc` on Linux
using `sbrk` at [https://danluu.com/malloc-tutorial/](https://danluu.com/malloc-tutorial/).)

Because of this complexity, there are many different implementations of `malloc`,
such as `jemalloc` or `TCMalloc`, which may be faster than the standard library
implementation.

In this analogy, Vulkan provides lower-level ingredients for memory allocation
on the GPU, which an application can use to implement its own GPU version of `malloc`.

For this tutorial, we won't have to implement our own `malloc` -- instead, we'll
use one of NVVK's built-in GPU memory allocators.

The code above `#include`s `nvvk::ResourceAllocatorDedicated`, which is a relatively
straightforward allocator that uses a single memory allocation per resource
(e.g. buffer, image, or acceleration structure).

!!! Tip Different Vulkan Memory Allocators
    NVVK also includes `nvvk::ResourceAllocatorDma` and `nvvk::ResourceAllocatorVma`.
    
    Unlike
    `nvvk::ResourceAllocatorDedicated`, these suballocate resources. Most production applications would use
    an allocator like one of these two -- instead of `nvvk::ResourceAllocatorDedicated` -- since
    using a dedicated allocation for every resource can run into an OS limit
    on the number of Vulkan dedicated allocations (as of this writing, this is 4096 on Windows; one can get
    this limit from `nvvk::Context`'s `VkPhysicalDeviceProperties::limits::maxMemoryAllocationCount`).
    
    Suballocators don't have this issue and can also be faster than using
    a dedicated allocation for every resource, although they are also more complex.
    
    Interested readers can find `ResourceAllocatorDma`'s implementation in
    `resourceallocator_vk.cpp`, `allocator_dma_vk.hpp`, and `memorymanagement_vk.hpp`; `ResourceAllocatorVma` uses
    an interface to the AMD Vulkan Memory Allocator.
    
    If one's interested in writing their own memory allocator, `nvvk::ResourceAllocatorDma`
    is a good place to start; they might also be interested in Chris Hebert's
    article [Vulkan Memory Management](https://developer.nvidia.com/vulkan-memory-management).

To set up an `nvvk::ResourceAllocatorDedicated`, add the following code after the
`assert(rtFeatures.rayQuery == VK_TRUE)` check:

~~~ cpp
  // Create the allocator
  nvvk::ResourceAllocatorDedicated allocator;
  allocator.init(context, context.m_physicalDevice);
~~~


!!! Tip Why does the `init` call work, when the first argument to `allocator.init` must have type `VkDevice`?
    `nvvk::Context` has an implicit cast to `VkDevice`, so that we don't have to
    write `context.m_device` -- see this line in `context_vk.hpp`:
    ~~~ cpp
    operator VkDevice() const { return m_device; }
    ~~~

Then deinitialize the allocator just before calling `context.deinit`:

~~~ cpp
  allocator.deinit();
~~~

## Vulkan Buffers

A `VkBuffer` (Vulkan buffer) is like a pointer to an array of bytes in C++.
However, when we create a `VkBuffer`, we also need to specify what the buffer
can be used for, using `VkBufferUsageFlagBits`.

Let's create a buffer to hold the rendered image. First, just after the include
directives, create constants for the width and the height of the image:

~~~ cpp
static const uint64_t render_width = 800;
static const uint64_t render_height = 600;
~~~

The buffer will contain a `float` for each of the red, green, and blue channels
for each of the `render_width` x `render_height` pixels in the image.

We'll eventually use the buffer as a [**storage buffer**](https://vulkan.lunarg.com/doc/view/1.2.135.0/windows/chunked_spec/chap13.html#descriptorsets-storagebuffer), which means that a program on the GPU can read and write it,
and as the destination for a transfer operation (the `vkCmdFillBuffer` call later
this chapter.)

Finally, we want the buffer's memory to be both **host visible** and **host coherent**.
This means that although the buffer exists on the GPU, we can read and write it
on the CPU by mapping it to CPU memory, and some parts of cache management will be
handled automatically. (We'll talk more about this in Section [Synchronization, Part 1: A Pipeline Barrier].)

Here's how to create a buffer. In Vulkan, one common pattern is to fill out a `CreateInfo` structure
that describes how to create an object, then to pass this structure to a function that creates the
object. (This helps with cases where objects might have lots of flags and settings.)
After calling `allocator.init`, compute the size of the buffer and create the buffer as follows:

~~~ cpp
  // Create a buffer
  VkDeviceSize bufferSizeBytes = render_width * render_height * 3 * sizeof(float);
  VkBufferCreateInfo bufferCreateInfo{.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO,
                                      .size  = bufferSizeBytes,
                                      .usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT};
  // VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT means that the CPU can read this buffer's memory.
  // VK_MEMORY_PROPERTY_HOST_CACHED_BIT means that the CPU caches this memory.
  // VK_MEMORY_PROPERTY_HOST_COHERENT_BIT means that the CPU side of cache management
  // is handled automatically, with potentially slower reads/writes.
  nvvk::Buffer buffer = allocator.createBuffer(bufferCreateInfo,                         //
                                               VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT       //
                                                   | VK_MEMORY_PROPERTY_HOST_CACHED_BIT  //
                                                   | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
~~~

!!! Tip
    We specify `VK_BUFFER_USAGE_STORAGE_BUFFER_BIT` because we'll use it as a
    storage buffer in Chapter [Compute Shaders], and
    `VK_BUFFER_USAGE_TRANSFER_DST_BIT` because we'll use it as the destination
    of a transfer operation in Chapter [Commands].

!!! Tip
    We'll have much more to say about these memory property flags later, but
    briefly, this makes it so that the CPU and GPU can access it quickly. Buffers
    that are only accessed by the GPU should instead use `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`. 

Make sure to destroy (deallocate) the buffer using the memory allocator before
the program ends by adding this just before `allocator.deinit()`:

~~~ cpp
  allocator.destroy(buffer);
~~~

You can run the program at this point, and it will allocate a 5.7 MB buffer on
the GPU, deallocate it, and exit. 

!!! Tip Why Use This Buffer Type?
    In this tutorial, we use a `VkBuffer` to store a 32-bit floating-point image.
    Many Vulkan applications use `VkImage`s to store images such as textures
    and rendered frames. In contrast to `VkBuffer`s, `VkImage`s can represent
    multidimensional data, image arrays and mipmaps, switching between layouts
    that are optimal for different purposes, interpolation using samplers, and
    more (though these can be emulated using `VkBuffers`, GPUs usually include
    dedicated hardware for these). They can also be drawn to the screen in real-time applications.
    
    However, `VkImage`s are also more complex than `VkBuffer`s, so we skip them
    in the main part of this tutorial.
    
    It is also unusual to use a 3-channel uncompressed 32-bit floating-point
    image; renderers will often use four channels, to support transparency, and
    16-bit floating-point, both for speed and to reduce memory usage. Using
    four channels would also allow the hardware to perform fewer store operations.
    However, this format choice saves several lines of code later.

## Retrieving Data from the GPU

Although we haven't written to the buffer yet, let's see how to transfer its
data from the GPU to the CPU.

After the call to `allocator.createBuffer`, add the following lines of code to
map the buffer to the CPU, interpret it as an array of `float`s, print out its
first three elements, and then return ownership back to the GPU:

~~~ cpp
  // In future chapters, we'll program the GPU to write to the buffer.
  // For now, let's read back undefined memory.

  // Get the image data back from the GPU
  void*  data    = allocator.map(buffer);
  float* fltData = reinterpret_cast<float*>(data);
  printf("First three elements: %f, %f, %f\n", fltData[0], fltData[1], fltData[2]);
  allocator.unmap(buffer);
~~~

Running the program should now print out something like
~~~ output
First three elements: 0.000000, 0.000000, 0.000000
~~~

We've just read **uninitialized data** back from the GPU since we haven't told the GPU to fill the buffer with anything. The buffer will most likely be filled with zeros, but it might also contain seemingly random values.

Let's now see how to set the buffer data to something other than 0!

!!! Tip Advice for Sharing Data Between the CPU and GPU
    In this main tutorial, we use a `HOST_VISIBLE | HOST_CACHED | HOST_COHERENT`
    buffer and access it from both the CPU and GPU, because it only requires one
    resource. From the perspective of CPU performance, the most important factor
    here is the `HOST_CACHED` flag, as without this flag each read of
    this buffer's memory from the CPU must go out to GPU memory and back,
    incurring latency (for PCIe, this can be hundreds of nanoseconds per read).
    
    However, computing on the GPU using `DEVICE_LOCAL` memory is usually faster.
    So in order to get better performance here, we might allocate two buffers:
    a buffer allocated from `DEVICE_LOCAL` memory, and a so-called **staging buffer**
    allocated from `HOST_VISIBLE | HOST_CACHED | HOST_COHERENT` memory.
    We would use the first buffer for
    GPU computation; then when we need to access its data from the CPU, we would
    use `vkCmdCopyBuffer` to copy its data to the staging buffer, and then map
    the staging buffer from the CPU.
    
    We'll implicitly use this method in Section [Uploading Data to the GPU],
    and will also show how to use this method in [the Images chapter of the extras](extras.html#images).

<!-- Chapter -->

# Commands

In this chapter, we'll create a command buffer, record a command to tell the GPU
to fill the buffer with the value `0.5`, submit the command buffer, and wait
for the GPU to finish.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/4_commands` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Command Buffers and Pools

A **command buffer** (`VkCommandBuffer`) holds a buffer of commands. You might
think of this as a list of instructions for the GPU to do, such as
"fill a buffer with a value", "copy an image", "resize an image",
"build a ray tracing acceleration structure", "bind a pipeline", or "trace rays".
Some commands, such as "fill a buffer with a value", perform actions, while others,
such as "bind a pipeline", change state affecting later actions.

To run a command buffer on the GPU, we submit it to a **queue**. We'll
only use a GCT (Graphics, Compute, and Transfer) queue in this tutorial, but
more complex setups are possible. In a full application, a GPU might perform
asynchronous transfers or compute data for future graphics operations on
different queues, in parallel with graphics operations executing on the primary queue.

Although command buffers contain their commands in order (this is part of [Vulkan submission order](https://vulkan.lunarg.com/doc/view/1.2.135.0/windows/chunked_spec/chap6.html#synchronization-implicit)),
since GPUs have thousands of cores which run in parallel 
(for instance, the GeForce RTX 3090 has 10,496 CUDA cores),
commands that perform actions start in order, but can run at the same time and complete
out of order. Vulkan provides tools for synchronization such as pipeline barriers,
fences, semaphores, and events, so that you can make
sure tasks complete in the order you want them to
(e.g. render the scene before denoising it).

First, add an `#include` for `<nvvk/error_vk.hpp>`. This includes the `NVVK_CHECK`
macro, which we use for API calls that can return error codes. It checks to see
if a Vulkan function returned the `VkResult` VK_SUCCESS` -- if it didn't,
it prints a message showing what failed and halts the program.

~~~cpp
#include <nvvk/error_vk.hpp>              // For NVVK_CHECK
~~~

!!! Tip
    Some Vulkan functions return `void` instead of `VKResult`, in which case we don't use `NVVK_CHECK`.

Applications get command buffers from command pools, so we'll need to
create a command pool. Add the following code after calling `allocator.createBuffer` to create a command pool:

~~~cpp
  // Create the command pool
  VkCommandPoolCreateInfo cmdPoolInfo{.sType            = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO,  //
                                      .queueFamilyIndex = context.m_queueGCT};
  VkCommandPool           cmdPool;
  NVVK_CHECK(vkCreateCommandPool(context, &cmdPoolInfo, nullptr, &cmdPool));
~~~

The `queueFamilyIndex` line here says that all command buffers allocated from this command pool
will use the GCT queue.

In the last line, a value of `nullptr` means that `vkCreateCommandPool`
will use the [default Vulkan CPU-side memory allocator](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#memory-allocation).

!!! Tip `nullptr` and `VK_NULL_HANDLE`
    We use `nullptr` when passing null pointers in structs, and `VK_NULL_HANDLE`
    for all **non-dispatchable Vulkan handles** (which are almost all handles).
    The reason is because these have different types with value 0: `nullptr` has
    a pointer type, while `VK_NULL_HANDLE`'s type depends on the platform
    since Vulkan 1.2.174.

Next, allocate a command buffer from the command pool. We can allocate multiple
command buffers at once, and create **secondary command buffers** (command buffers
that other command buffers can include), but we only need one primary-level
(i.e. top-level) command buffer.

~~~cpp
  // Allocate a command buffer
  VkCommandBufferAllocateInfo cmdAllocInfo{.sType              = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,
                                           .commandPool        = cmdPool,
                                           .level              = VK_COMMAND_BUFFER_LEVEL_PRIMARY,
                                           .commandBufferCount = 1};
  VkCommandBuffer             cmdBuffer;
  NVVK_CHECK(vkAllocateCommandBuffers(context, &cmdAllocInfo, &cmdBuffer));
~~~

Let's begin recording the command buffer, by calling `vkBeginCommandBuffer`.
We'll tell Vulkan that we'll only use the command buffer once -- we won't submit
it twice without rerecording it.

~~~cpp
  // Begin recording
  VkCommandBufferBeginInfo beginInfo{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
                                     .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
  NVVK_CHECK(vkBeginCommandBuffer(cmdBuffer, &beginInfo));
~~~

Now, let's record a command for the GPU to fill the `VkBuffer` `buffer` with
32 repeating bits of data! The declaration for the `vkCmdFillBuffer` command
is:

~~~ cpp
void vkCmdFillBuffer(
    VkCommandBuffer                             commandBuffer,
    VkBuffer                                    dstBuffer,
    VkDeviceSize                                dstOffset,
    VkDeviceSize                                size,
    uint32_t                                    data);
~~~

So, to fill the entire buffer with the 32 bits corresponding to the float 0.5,
write:

~~~ cpp
  // Fill the buffer
  const float fillValue = 0.5f;
  const uint32_t& fillValueU32 = reinterpret_cast<const uint32_t&>(fillValue);
  vkCmdFillBuffer(cmdBuffer, buffer.buffer, 0, bufferSizeBytes, fillValueU32);
~~~

(`buffer` is an `nvvk::Buffer` containing both a `VkBuffer` and its underlying
memory allocation, so we use `buffer.buffer` to get the `VkBuffer`. The
`reinterpret_cast` line references the bits of `fillValue` as a `uint32_t`. )

There's one more thing to do before the end of the command buffer: We need to
add a command to make sure the CPU can read what the GPU writes!

### Synchronization, Part 1: A Pipeline Barrier

Here are two important aspects of programming in Vulkan:

1. Starting two commands in sequence does not necessarily mean that the first
   command will be followed by the second command. They will start in [submission order](https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#synchronization-submission-order),
   but they may run in parallel or take turns executing.

2. Unlike C++, running a command that writes to memory doesn't
   mean that subsequent commands will be able to automatically read those written values from that memory except
   in special cases. (In other words, the Vulkan spec notes that
   "happens-before is not always sufficient for a write to be visible to a read.
   Additional availability and visibility operations may be required for
   writes to be visible-to other memory accesses.")

Pipeline barriers synchronize memory, and they are one type of thing that makes it
so that pipeline stages (such as transfers, shaders, or the CPU) can wait for other pipeline stages'
memory accesses and read what those pipeline stages have completed writing.

Pipeline barriers can be one of the most complex parts of the Vulkan API,
so we've added options so that readers can choose how deeply they want
to read about pipeline barriers. On a first read-through, a reader
might skip the text boxes below, while more advanced readers and readers
on subsequent read-throughs might be interested in the additional detail
and resources.

In some sense, pipeline barriers say
"Make sure that these memory operations finish before these other
tasks start". One can almost visualize it like a physical barrier in the command
buffer blocking progress on some types of memory operations until other tasks
have finished.

In our case, we want
to say "Make it so that the writes from the `vkCmdFillBuffer` call finish
and make sure the CPU will be able to read them."

We can do this using a `VkMemoryBarrier`, which synchronizes memory across
the entire GPU. Since `vkCmdFillBuffer` is a **transfer operation** (this
comes from the documentation for `vkCmdFillBuffer`), we say "Make host (CPU)
reads wait for transfer writes".

Add the following code after the `vkCmdFillBuffer` call:

~~~ cpp
  // Add a command that says "Make it so that memory writes by the vkCmdFillBuffer call
  // are available to read from the CPU." (In other words, "Flush the GPU caches
  // so the CPU can read the data.") To do this, we use a memory barrier.
  VkMemoryBarrier memoryBarrier{.sType         = VK_STRUCTURE_TYPE_MEMORY_BARRIER,
                                .srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,  // Make transfer writes
                                .dstAccessMask = VK_ACCESS_HOST_READ_BIT};      // Readable by the CPU
  vkCmdPipelineBarrier(cmdBuffer,                                               // The command buffer
                       VK_PIPELINE_STAGE_TRANSFER_BIT,                          // From the transfer stage
                       VK_PIPELINE_STAGE_HOST_BIT,                              // To the CPU
                       0,                                                       // No special flags
                       1, &memoryBarrier,                                       // An array of memory barriers
                       0, nullptr, 0, nullptr);                                 // No other barriers
~~~

Interested readers can read on further to learn more about pipeline barriers.

!!! Tip Vulkan Pipeline Stages
    Pipeline barriers synchronize different sets of **pipeline stages**, which `vkCmdPipelineBarrier`
    uses in its second and third arguments, `srcStageMask` and `dstStageMask`
    
    A pipeline stage represents a place where data manipulation happens, with
    two exceptions*. For example, there is a pipeline stage for the
    compute shader, a pipeline stage for the host (CPU-side system),
    and a pipeline stage for transfer operations, such as data copies.
    
    A pipeline stage is represented by a `VkPipelineStageFlagBits` bitmask.
    Multiple pipeline stages can be combined into a `VkPipelineStageFlags`
    value using the bitwise OR operator `|`.
    
    The `srcStageMask` and `dstStageMask` `VkPipelineStageFlags` arguments
    in `vkCmdPipelineBarrier` essentially say "Depending on some details
    in the barriers, stages in `dstStageMask` will wait for and be able
    to read the writes of stages in `srcStageMask`."
    
    *These exceptions are the "top of pipe" and "bottom of pipe" pipeline
    stages. The name "pipeline" comes from the graphics pipeline, where
    work flows from vertex input through geometry processing, shading,
    and data output, in a pipelined fashion. The `TOP_OF_PIPE` and
    `BOTTOM_OF_PIPE` bits represent the start and end of an imaginary
    pipeline from `TOP_OF_PIPE`, to all of the stages that the queue
    can perform, to `BOTTOM_OF_PIPE`.

!!! Tip Vulkan Access Flags and `nvvk::makeAccessMaskPipelineStageFlags`
    `srcAccessMask` and `dstAccessMask` in `VkMemoryBarrier` are
    `VkAccessFlags` bitmasks. A single `VkAccessFlagBits` bitmask
    represents a way that memory can be accessed. Some examples
    are `VK_ACCESS_TRANSFER_WRITE_BIT` (memory writes by
    the transfer stage), `VK_ACCESS_HOST_READ_BIT` (memory reads
    by the host, i.e. CPU-side system), and `VK_ACCESS_SHADER_WRITE_BIT`
    (memory writes by shaders).
    
    Like pipeline stages, `VkAccessFlagBits` values can be
    combined using bitwise OR to create a combined `VkAccessFlags` value.
    
    In many cases (usually with the exception of `TOP_OF_PIPE`
    and `BOTTOM_OF_PIPE`), it's possible to infer pipeline stages from
    access flags. (For instance, `VK_ACCESS_TRANSFER_WRITE_BIT` is part
    of the `VK_PIPELINE_STAGE_TRANSFER_BIT` pipeline stage.)
    `nvvk::makeAccessMaskPipelineStageFlags`, located
    in NVVK's `commands_vk.hpp` is a helper for this; given a
    `VkAccessFlags` access mask, it derives compatible stage flags.
    We'll see it in use in [the extras](extras.html#images).

!!! Tip How Pipeline Stages and Accesses Differ
    One of the reasons that pipeline barriers are complex is because
    they involve both **execution dependencies** (operation set A must
    happen before operation set B), and **memory dependencies** (one
    set of memory accesses by operation set A must be made available,
    and all available memory writes must be made visible to a specified
    set of memory accesses by operation set B.)
    
    Pipeline stages are used in execution dependencies, but both
    pipeline stages and access scopes are used in memory dependencies.
    
    As mentioned above, the sets of pipeline stages needed can often
    be inferred from the combined access scopes and
    `nvvk::makeAccessMaskPipelineStageFlags`.
    We'll see this in [the extras](extras.html#images).

!!! Tip What Pipeline Barriers Don't Synchronize
    Note that pipeline barriers only synchronize things occurring on the
    GPU, in addition to memory operations within the GPU and involving
    other systems; since they're recorded commands in a command buffer
    submitted later on, they don't prevent the CPU from running further
    commands (at least in the normal sense).
    
    To make the CPU wait for the GPU, we'll use `vkQueueWaitIdle` later;
    we could also use `VkFence`s. (`vkQueueWaitIdle` is equivalent to
    waiting on a fence.)
    
    Unfortunately, `vkQueueWaitIdle` isn't guaranteed
    to be enough to make memory writes from the GPU correctly accessible
    by the CPU; see the last text box in this section for a full explanation.
    However, the reverse (making memory writes by the CPU accessible by
    the GPU) often doesn't require a pipeline barrier, because of the
    language in section 6.9 of the [Vulkan specification](https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#synchronization-submission-host-writes),
    "Host Write Ordering Guarantees".
    
    At this point in the program, the GPU is waiting for the CPU to submit work,
    while the CPU is recording a command buffer.

!!! Tip Types of Barriers
    There are three types of barriers supported by `vkCmdPipelineBarrier`.
    
    **Memory barriers** say "Create a memory dependency between these
    access scopes, applied to all of memory." They're
    represented using [VkMemoryBarrier](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkMemoryBarrier.html)
    structures.

    **Buffer memory barriers** say "Create a memory dependency between
    these access scopes, applied to this range of memory."
    and they can also transfer ownership of a buffer between queues.
    They're
    represented using [VkBufferMemoryBarrier](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkBufferMemoryBarrier.html)
    structures. Unless one needs to perform a queue ownership
    transfer, using a memory barrier instead is fine.
    
    Finally, **image memory barriers** say "Create a memory dependency
    between these access scopes, applied to this subresource range
    of an image" (since images are complex, this is more complex than
    specifying a linear range of memory). In addition to transferring
    ownership between queues, they can also perform **image layout transitions**
    between different **image layouts**. We'll describe this more in
    [the extras](extras.html#images). They're represented using
    [VkImageMemoryBarrier](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageMemoryBarrier.html).

!!! Tip Memory Barrier Performance
    On the current NVIDIA GPU driver, there's currently no benefit from
    using buffer memory bariers over regular memory barriers. When it
    comes to image memory barriers, which are discussed in [the extras](extras.html#images)
    picking precise layouts is not required for optimal performance either
    -- `VK_IMAGE_LAYOUT_GENERAL` works fine, and layout transitions don't
    exist on the GPU hardware (see slide 66 of [this deck](https://www.slideshare.net/tlorach/siggraph-2016-vulkan-and-nvidia-the-essentials)).
    
    However, an application should use as few barriers as possible
    (including combining barriers into a single barrier and removing
    redundant barriers), since barriers can cause a GPU to flush the
    entire pipeline.
    For more information, see the Barriers section of Nuno Subtil,
    Matthew Rusch, and Ivan Fedorov's article [Tips and Tricks: Vulkan Dos and Don'ts](https://developer.nvidia.com/blog/vulkan-dos-donts/).

<!-- Custom admonition to handle diagrams -->
<div class ="admonition tip">
<div class="admonition-title">Technical Note: More About This Pipeline Barrier</div>
In some sense, this pipeline barrier is really about cache invalidation.
If one's worked with multithreading in C++, they might be familiar with
how caches sometimes need to be considered when sharing information between different CPU threads --
this issue is similar, except it's between the CPU and GPU instead of
two CPU threads.

Both the CPU and GPU have caches in their memory system. Here's a
simplified diagram:

***************************************************************
*
*             .-----------PCI Express-----------.
*            |                                   |
* .---------------------.             .---------------------.
* |        Cache        |             |        Cache        |
* '---------------------'             '---------------------'
*            |                                   |
*            |                                   |
* .---------------------.            .-----------------------.
* | Host (CPU-side) RAM |            | Device (GPU-side) RAM |
* '---------------------'            '-----------------------'
*            |                                   |
*            |                                   |
* .---------------------.             .---------------------.
* |   Data processing   |             |   Data processing   |
* |  (and more caches)  |             |  (and more caches)  |
* '---------------------'             '---------------------'
*
***************************************************************

Later, we'll call `vkQueueWaitIdle`, which will make sure that
GPU writes are flushed to each GPU's VRAM. But it won't necessarily flush the
writes to the top-right cache, which would make the memory **available** to
the CPU (i.e. make it so that the CPU can see the correctly
updated contents of the buffer.)

***************************************************************
*
*             .-----------PCI Express-----------.
*            |                                   |
* .---------------------.             .---------------------.
* |        Cache        |             |        Cache        |
* '---------------------'             '---------------------'
*            |                                   | !
*            |                                   |
* .---------------------.            .-----------------------.
* | Host (CPU-side) RAM |            | Device (GPU-side) RAM |
* '---------------------'            '-----------------------'
*            |                                   ^
*            |                                   |
* .---------------------.             .---------------------.
* |   Data processing   |             |   Data processing   |
* |  (and more caches)  |             |  (and more caches)  |
* '---------------------'             '---------------------'
*
***************************************************************

This pipeline barrier tells the GPU to flush the writes to the
top-right cache, making the memory available to the CPU.

Now we can more precisely say what setting the
`VK_MEMORY_PROPERTY_HOST_COHERENT_BIT` flag when creating the buffer did:
this makes it so that these writes are also automatically
made **visible** to the CPU, through invalidating the
cache at the top-left.

Now that the system made the memory
available and visible, the CPU can read it correctly.

In practice, the driver implementation might always flush the
right caches, and so the application might work without this
pipeline barrier.
</div>

!!! Tip More Resources About Pipeline Barriers
    We'll describe image pipeline barriers more in [the extras](extras.html#images).
    
    Here are some other resources for readers interested in learning more about pipeline barriers.
    Note that many of the concepts in pipeline barriers are useful in other forms
    of Vulkan synchronization, such as events and render passes:
    
    * The [Vulkan Specification](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#synchronization-dependencies) 
      describes pipeline barriers in the "Execution and Memory Dependencies"
      section. Chapter 6, "Synchronization and Cache Control",
      also describes other forms of synchronization in Vulkan.
    
    * Hans-Kristian Arntzen's [Yet another blog explaining Vulkan synchronization](https://themaister.net/blog/2019/08/14/yet-another-blog-explaining-vulkan-synchronization/)
      is an in-depth look at Vulkan synchronization with a focus on pipeline
      barriers.
    
    * Nuno Subtil, Matthew Rusch, and Ivan Fedorov's [Tips and Tricks: Vulkan Dos and Don'ts](https://developer.nvidia.com/blog/vulkan-dos-donts/)
      includes a section on barrier performance in Vulkan.
    
### Ending and Submitting the Command Buffer

Now, end recording of the command buffer:

~~~ cpp
  // End recording
  NVVK_CHECK(vkEndCommandBuffer(cmdBuffer));
~~~

Finally, submit the command buffer, so that the GPU can start working on it.
We could submit multiple command buffers at once, but here we'll only submit one.

~~~ cpp
  // Submit the command buffer
  VkSubmitInfo submitInfo{.sType              = VK_STRUCTURE_TYPE_SUBMIT_INFO,  //
                          .commandBufferCount = 1,                              //
                          .pCommandBuffers    = &cmdBuffer};
  NVVK_CHECK(vkQueueSubmit(context.m_queueGCT, 1, &submitInfo, VK_NULL_HANDLE));
~~~

!!! Tip `vkQueueSubmit` Performance
    All submits are managed by the operating system, which makes this call take
    a bit longer than other Vulkan calls (usually on an order of tens of microseconds).
    Therefore, applications should usually batch multiple submits into one
    when possible.

Before reading the data back from the GPU, or freeing the command buffer we
allocated, we need to make sure the GPU has finished processing the command buffer.
(Imagine if you were trying to follow a recipe, but someone took away the
instructions midway through the process!) We'll describe how to do that next.

## Synchronization, Part 2: `vkQueueWaitIdle`

One of the forms of synchronization in Vulkan is to make the CPU wait
until the GPU has finished all the work on a queue. It's not the most efficient
-- the CPU could work on other things in parallel instead, possibly using Vulkan semaphores or fences
-- but it's good enough for now.

To make the CPU wait for the GCT queue to finish, add the following lines
after the `vkQueueSubmit` call:

~~~ cpp
  // Wait for the GPU to finish
  NVVK_CHECK(vkQueueWaitIdle(context.m_queueGCT));
~~~

## Freeing Command Buffers and Pools

Make sure to free the command buffer just before calling `allocator.destroy(buffer)`.
As with many Vulkan functions, we could free many command buffers at once, but
here we only have one to free:

~~~ cpp
  vkFreeCommandBuffers(context, cmdPool, 1, &cmdBuffer);
~~~

Finally, destroy the command pool:

~~~ cpp
  vkDestroyCommandPool(context, cmdPool, nullptr);
~~~

A value of `nullptr` here means that `vkDestroyCommandPool` will use the
default deallocator, like how we created the command pool.

Now, you can run the program and see that you've successfully run a command
buffer on the GPU that changed the contents of `buffer`!

~~~ output
First four elements: 0.500000, 0.500000, 0.500000, 0.500000
~~~

## Summary

In the last two chapters, we saw how to:

* Allocate GPU memory using `nvvk::ResourceAllocatorDedicated`;
* Create a `VkBuffer` representing an image;
* Map data from the GPU to the CPU;
* Create a command buffer and a command pool;
* Record and submit a command buffer with a `vkCmdFill` command.



<!-- Chapter -->
# Writing an Image

In this chapter, we'll show how to write a GPU-rendered image to disk in
a format that other programs can read. That way, you can visualize
what your program renders.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/5_write_image` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.
    
## Including Additional Headers

For our image, we'll use [stb_image_write](https://github.com/nothings/stb/blob/master/stb_image_write.h)
to save our image to a [Radiance High Dynamic Range](https://en.wikipedia.org/wiki/RGBE_image_format) (HDR) file.

HDR files store floating-point image data with a larger range of colors than 8-bit RGB. They can be read by
many different tools, such as Photoshop and the NVIDIA Texture Tools Exporter.
Most importantly, though, we can save linear floating-point image data with one line of code using
`stbi_write_hdr` (after including the stb_image_write library).

At the beginning of `main.cpp`, add the following two lines to include stb_image_write, as well as the
implementation of stb_image_write (since stb_image_write is a single-file header-only library):

~~~ cpp
#define STB_IMAGE_WRITE_IMPLEMENTATION
#include <stb_image_write.h>
~~~

## Writing an HDR File

To write an HDR file from the data in `buffer`, remove these two lines that printed out
the first four floating-point elements of the buffer:

~~~ cpp
  float* fltData = reinterpret_cast<float*>(data);
  printf("First three elements: %f, %f, %f\n", fltData[0], fltData[1], fltData[2]);
~~~

In their place, add the following line:

~~~ cpp
  stbi_write_hdr("out.hdr", render_width, render_height, 3, reinterpret_cast<float*>(data));
~~~

This tells stb_image_write to write an HDR image with:
* filename `out.hdr`,
* dimensions `render_width` x `render_height`,
* three channels,
* and the floating-point data in `data`.

You should now be able to compile and run the program. It should create a file
named `out.hdr` in the CMake directory for the project, which will be something
like `vk_mini_path_tracer/cmake_build/5_write_image`. If you open the data
in a supported image viewer, you should be able to see a flat gray image.

![The output of this chapter's program.](images/4-gray.png)

In the next two chapters, we'll show how to write a shader -- an arbitrary program that can
be run on the GPU -- to render an image, instead of using `vkCmdFillBuffer`!

!!! Tip Technical Note: These Files Use Linear RGB with the sRGB Primaries and White Point
    The HDR files that `stbi_write_hdr` creates use linear RGB colors, while most image
    editors use non-linear sRGB by default. This means that although we wrote a linear color of
    (0.5, 0.5, 0.5), most image editors will list an sRGB color of (188/255, 188/255, 188/255).
    
    It's not necessary to know about this for this tutorial since this tutorial never
    uses sRGB. However, for interested readers, a short explanation is that linear
    colors are directly proportional to amounts of light. Linear colors should
    almost always be used when mathematically modeling how light interacts with surfaces.
    
    However, sRGB colors use a curve that sort of mimics human perception of values,
    which is non-linear. Because of this, sRGB colors
    are often used for 8-bit-per-channel image storage -- so the image editor above
    is listing an sRGB color between 0 to 1, multiplied by 255 (the largest
    unsigned 8-bit number).
    
    More specifically, these linear colors use the same red, green, and blue primaries
    as `sRGB`, and linear color (1, 1, 1) is the same as sRGB (1, 1, 1) (they have the
    same white point). But linear values are proportional to light emitted, while
    sRGB uses a non-linear encoding of light that is better for 8-bit quantization.
    
    The full formula to convert from an sRGB color in the range from 0 to 1 to a
    linear color in the range from 0 to 1 is
    
    $$v_{\text{sRGB}} = {\begin{cases} \frac{v_{\text{linear}}}{12.92} & \text{if } v_{\text{linear}} \leq 0.04045 \\ \left(\frac{v_{\text{linear}} + 0.055}{1.055}\right)^{2.4} & {\text{otherwise}} \end{cases}}.$$
    
    Working with color spaces such as linear (Rec. 709) RGB and sRGB is part of the field of color science, which has many surprising nuances and recent developments with regards to computer graphics. For instance, some rendering engines are experimenting with using "sharp" color primaries other than the Rec. 709 primaries to better model light reflection, while others are attempting to simulate lighting using a full spectrum of colors (instead of only three primaries).
    
    (The HDR file format doesn't specify an exact color space; it was designed
    before sRGB and the underlying Rec. 709 standard were introduced.)

<!-- Chapter 6 -->
# Compute Shaders

**Shaders** are like functions that a GPU can run. They provide one way to program
your GPU to do things from simulating protein folding, to rendering virtual
worlds, to modeling the weather. Eventually, we'll write a shader that will
perform many iterations of path tracing for each pixel in an image.

In this chapter, we'll write a Hello World-like **compute shader**. Then we'll
use Vulkan to run it on 128 **invocations** (threads) in parallel.

The shaders in this tutorial use the OpenGL Shading Language (GLSL). This is
a language with a C++-like syntax designed for GPUs.

As an example of GLSL, here's the complete compute shader that we'll write in
this chapter. We'll explain what this does as we go through the chapter:

~~~ glsl linenumbers
#version 460
#extension GL_EXT_debug_printf : require

layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;

void main()
{
  debugPrintfEXT("Hello from invocation (%d, %d)!\n", gl_GlobalInvocationID.x, gl_GlobalInvocationID.y);
}
~~~

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/6_compute_shader` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Using Compute Shaders for Ray Tracing

<!-- We use the terminology from https://nvpro-samples.github.io/vk_raytracing_tutorial_KHR/: ray generation, closest hit, any hit, ray tracing. -->

In this tutorial, we'll use a compute shader with **ray queries** to perform what
might usually be done using **ray tracing shaders**. This may be somewhat surprising,
but it saves about 75 lines of code!

However, there are tradeoffs involved, and
a later chapter will describe how to perform ray tracing using the ray tracing pipeline.
The rest of this section is optional, and readers can skip to the Writing a Hello, World Compute Shader section.

There are many different types of shaders.

* **Compute shaders** are designed for general-purpose computation
  (they're sort of like the interiors of parallel `for` loops in C++, 
  with some exceptions). GPUs run them using **compute pipelines**.

* **Ray generation**, **any hit**, **callable**, **closest hit**, **intersection**,
  and **miss** shaders are used with **shader binding tables** (SBTs) for ray tracing. GPUs
  run them using **ray tracing pipelines**.

* **Vertex**, **tessellation control**, **tessellation evaluation**, **geometry**,
  **fragment**, **task**, and **mesh** shaders are part of **graphics pipelines**.
  Applications use them for things such as rasterization (although there are many
  other uses as well).

These can all be thought of as different ways to operate on different types of data.
As a result, one can often use one shader type to do the task of another.

In a traditional tutorial using the ray tracing pipeline for ray tracing, we would
use a ray generation shader that takes a pixel coordinate and traces multiple rays,
a closest hit shader that takes a ray-mesh intersection and returns the properties
of a mesh at the intersection, and a miss shader that takes a direction and returns
the color of the sky in that direction. We would then create a shader binding table
with records for each of these shaders, and use a ray tracing pipeline to launch rays.

However, ray queries allow us to perform ray tracing not only from ray tracing
pipelines, but also from graphics and compute pipelines. All our different
**materials** (describing how light interacts with the sky and the mesh) are included
in a single shader, which reduces the amount of code to write.

!!! Tip For advanced readers: Tradeoffs between ray queries and ray tracing pipelines
    There are some challenges to overcome
    when using ray queries for larger projects. Please see the [NVIDIA Ray Tracing
    Tutorials](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR) for how
    to implement ray tracing pipelines:

    * The GPU lays out threads in a ray generation shader in a more cache-friendly
      way, which also maps well to how the driver stores Vulkan images internally.
    
    * Only ray tracing pipelines support callable shaders, which allow shaders to call other shaders (even recursively, so
      long as one provides the maximum recursion depth in advance so that the implementation can allocate enough stack size).
      
    * Ray tracing pipelines use a fixed ruleset for handling networks of shaders,
      which allows for further hardware optimizations. For instance, this can significantly
      help in scenes that use large numbers of shaders for different materials.
    
    However, for this tutorial, where we only have one material type, a ray query
    approach can run at about the same speed as a ray tracing pipeline approach as of this writing.
    
    One can always combine ray queries and the ray tracing pipeline in different
    parts of a renderer. For instance, one could use ray queries in a compute shader to
    compute ambient occlusion, while using the ray tracing pipeline for reflections
    or global illumination.

## Writing a Hello, World Compute Shader

### Work Groups

Informally, a GPU invokes the entry point of a compute shader for every point in a 3D grid.
The application defines how to divide this grid into 3D **work groups** of equal size; each
work group contains a 3D grid of invocations.

For example, imagine we wanted to multiply each value in a 15 x 13 array of `float`s by 2.
One way to do this would be to tile the array with work groups each with 16 x 8 x 1 invocations.
This would require a grid of 1 x 2 x 1 work groups -- using one dot per invocation:

********************************************************************************
*
* Array (15 x 13)                 Grid of work groups (1 x 2 x 1), (16 x 8 x 1)
* 
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * * (0, 0, 0) to
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * * (15, 7, 0)
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *   * * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *   o o o o o o o o o o o o o o o o
* * * * * * * * * * * * * * * *   o o o o o o o o o o o o o o o o
* * * * * * * * * * * * * * * *   o o o o o o o o o o o o o o o o
* * * * * * * * * * * * * * * *   o o o o o o o o o o o o o o o o (0, 8, 0) to
* * * * * * * * * * * * * * * *   o o o o o o o o o o o o o o o o (15, 15, 0)
*                                 o o o o o o o o o o o o o o o o
*                                 o o o o o o o o o o o o o o o o
*                                 o o o o o o o o o o o o o o o o
*
********************************************************************************

Each element would check to make sure that it's within the bounds of the array before accessing memory.

!!! Tip
    One could also number the 15 x 13 = 195 elements of the array from 0 to 194,
    then run a compute shader that converts between 1D and 2D indexing systems,
    using $\text{ceil}(195/128) = 2$ work groups of 128 x 1 x 1 invocations.

!!! Tip Choosing Work Group Sizes: Multiples of 32 or 128 Threads
    An NVIDIA GPU runs a compute pipeline by dividing each workgroup into
    **subgroups** (warps) of 32 invocations. This is why it is a good idea
    to make the number of invocations in a work group a multiple of the
    subgroup size, 32.
    
    Each 32-invocation subgroup runs
    using a SIMT execution model: All invocations in a subgroup process the
    same instructions at the same time. (To handle things such as branches,
    subgroups use a bitmask to keep track of which invocations are active
    at any instruction.)
    
    However, each **Streaming Multiprocessor** (SM) can handle multiple
    work groups at once and multiple subgroups per work group with
    limits on each, which means that it's often better to have multiple
    subgroups per work group. (In the case of this sample, moving from
    a work group size of 8 x 4 x 1 to a work group size of 16 x 8 x 1
    gave a significant speedup). The optimal work group size depends on
    many other factors as well, such as how many **registers** and how
    much **shared memory** the shader uses.
    
    An excellent way to calculate the optimal work group size on a GPU
    without needing performance measurements is to use the continually
    updated [CUDA Occupancy Calculator](https://docs.nvidia.com/cuda/cuda-occupancy-calculator/index.html).
    
    By setting the registers per thread and shared memory per block in
    box 2, we can see how the block (work group) size affects the
    occupancy (number of warps that can run at once per SM, which
    we want to be as high as possible).
    
    In this case, setting the
    "Registers Per Thread" to, say, 32 (this will depend on the shader,
    and can be queried using Vulkan code such as used in `nvvk::nvprintPipelineStats`,
    or a tool such as Nsight Graphics)
    and the "User Shared Memory Per Block (bytes)" to 0 (although it's
    useful we don't use shared memory in this tutorial):
    
    ![Box 2 settings for this tutorial in the CUDA Occupancy Calculator.](images/5-occupancyCalculator.png)
    
    will show this graph in the top-middle:
    
    ![Occupancy vs. block size for a compute capability 8.6 GPU](images/5-oc-cc-8-6.png)
    
    We can see that for [compute capability](https://developer.nvidia.com/cuda-gpus) 8.6
    GPUs (each compute capability roughly corresponds to a GPU architecture - CC 8.6 includes
    Ampere GeForce GPUs, for instance), block sizes of 96, 128, 192, 256, ... threads
    are expected to give the highest performance.
    
    But what if we want to run on a Turing RTX GPU (e.g. an RTX 2070)? In this case, setting
    the "Select Compute Capability (click)" field in Box 1 to 7.5 (the RTX 2070's compute
    capability) gives this graph:
    
    ![Occupancy vs. block size for a compute capability 7.5 GPU](images/5-oc-cc-7-5.png)
    
    This says that block sizes of 64, 128, 256, .. threads are expected to give the
    highest performance. So, a block size of 128 threads should work well across
    different RTX GPUs. Indeed, using 128 threads per subgroup makes ray tracing run
    faster than using, say, 32 threads per subgroup -- and all we had to do was
    change the work group size!

### An Empty Compute Shader

Our compute shader will use 1 workgroup of 16 x 8 x 1 invocations, and each
invocation will use the new `GL_EXT_debug_printf` extension to print out "Hello, world!".

To start out, create a new directory, named `shaders`, next to your project's `main.cpp` file.
Then create an empty file inside `shaders` named `raytrace.comp.glsl`. This will contain our compute shader code.
(It won't do ray tracing in this chapter, but it will in later chapters.)

Now, use CMake to re-configure and re-generate the project. This sets up the
steps to compile this shader. If you're using Visual Studio,
you should now see `raytrace.comp.glsl` appear in the `Shader Files` filter:

![](images/5-vsShader.png)

Open `raytrace.comp.glsl`, and enter in the following code. This

* uses the `#version` preprocessor command to say that the file uses GLSL version 4.6
* specifies the size of this compute shader's work groups (16 x 8 x 1)
* defines a function, `main`, that does nothing:

~~~
#version 460

layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;

void main()
{
}
~~~

You should now be able to compile the project, and see it generate a **SPIR-V** file
named `raytrace.comp.glsl.spv` next to `raytrace.comp.glsl`. The build system has translated
this GLSL code into the SPIR-V intermediate representation, which Vulkan will use.
We'll talk about what this representation in the How Shader Compilation Works section.

### A Hello, World! Compute Shader

For now, we'll show how to use the `debugPrintfEXT` function to print text. To do
this, we'll add a preprocessor directive to require the `GL_EXT_debug_printf` extension,
then call `debugPrintfEXT` inside `main()`:

~~~
#version 460
#extension GL_EXT_debug_printf : require

layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;

void main()
{
  debugPrintfEXT("Hello world!");
}
~~~

When the GPU runs the compute shader, all 128 invocations will print "Hello world!".
The validation layer will then relay these messages to the CPU, and finally to
the console window.

We'll give more details about how this message relay in `debugPrintfEXT` works below,
and you can also visit [this page](https://github.com/KhronosGroup/Vulkan-ValidationLayers/blob/master/docs/debug_printf.md)
for more information.

## How Shader Compilation Works

On the CPU side, you might be familiar with how CPUs have a variety of different **architectures**, such
as x86, x64, and ARM. Each architecture has its own **instruction set** (which may also
change over time). As a result, you usually must compile your program for each
architecture (e.g. maintaining 32-bit, 64-bit, and ARM versions of applications).

You might also be familiar with tools that use **intermediate representations**, like LLVM.
For instance, one might compile a C++ program using Clang, which translates the C++ code
into LLVM's intermediate representation; LLVM then optimizes this code, and can then
output machine code for the target architecture.

GPUs, on the other hand, have a much wider variety of architectures and instruction sets.
Yet GLSL shader writers can usually ignore this, and only must compile a shader once.

To do this, Vulkan uses a similar approach as LLVM: A program such as `glslangValidator`
translates GLSL code to Standard Portable Intermediate Representation-V format (SPIR-V).
When creating a compute shader pipeline, your GPU's driver takes this SPIR-V representation
and optimizes and compiles it to produce GPU-specific assembly code.

************************************************
*
*    Clang with C++        Vulkan with GLSL
*  
*     .----------.          .-----------.
*     | C++ code |          | GLSL code |
*     '----------'          '-----------'
*          |                      |
*          |                      |
*    Clang frontend        glslangValidator
*          |                      |
*          v                      v
*     .---------.            .--------.
*     | LLVM IR |            | SPIR-V |
*     '---------'            '--------'
*          |                      |
*          |                      |
*        LLVM                GPU driver
*          |                      |    
*          v                      v
*  .--------------.        .--------------.
*  | CPU assembly |        | GPU assembly |
*  '--------------'        '--------------'
*
***********************************************

The main difference is that final compilation and optimization occurs in a just-in-time
fashion (when creating the pipeline) in Vulkan, instead of at compile-time. Once a
shader has been converted to GPU assembly, it's cached on the system.

This means
that the developer doesn't have to compile shaders for every GPU architecture in advance,
and that the compiler can apply optimizations that it can only do if it knows some things
about the pipeline, but also means that programs with, say, tens of thousands of shaders can
face challenges with runtime compilation.

!!! Tip Shader Assembly
    Interested in seeing what compiled GPU assembly code looks like? CUDA includes
    publicly available tools for disassembling CUDA kernels to assembly, and 
    one can view examples of shader assembly and lists of instructions at
    https://docs.nvidia.com/cuda/cuda-binary-utilities/index.html#cuobjdump.

In this sample, the build system used a command such as

```
glslangValidator.exe --target-env vulkan1.2 -o shaders/raytrace.comp.glsl.spv shaders/raytrace.comp.glsl
```

to generate SPIR-V. The CMake build system will do this automatically for any GLSL files in
the project. (However, one does need to re-run CMake in order for it to detect new shader files.)

!!! Tip CMake Details: How the Build System Detects and Re-Translates GLSL Files
    Here's some more information for interested readers about how the build system implements GLSL translation.
    
    Each checkpoint's CMakeLists.txt file (e.g. `_edit/CMakeLists.txt`) has a few lines that look like
    this:
    
    ~~~
    file(GLOB_RECURSE GLSL_SOURCE_FILES "shaders/*.glsl")
    foreach(GLSL ${GLSL_SOURCE_FILES})
        get_filename_component(FILE_NAME ${GLSL} NAME)
        _compile_GLSL(${GLSL} "shaders/${FILE_NAME}.spv" GLSL_SOURCES SPV_OUTPUT)
    endforeach(GLSL)
    ~~~
    
    When running CMake to configure the project, these lines find all of the GLSL files in the `shaders`
    subdirectory (line 1), then adds commands to compile each one (line 3, which calls _compile_GLSL).
    This is why one needs to re-run CMake (re-running these lines) in order for the build system to find
    new GLSL files.
    
    The `_compile_GLSL` CMake macro is defined in `nvpro_core/setup.cmake`, which is a
    CMake file used by almost all NVIDIA DesignWorks samples.. It ultimately calls CMake's
    `add_custom_command` to add a build rule to run `glslangValidator` on the given GLSL source files.

!!! Tip Inspecting SPIR-V
    Interested readers can find out what the SPIR-V code for this shader looks like using `spirv-dis`!

    `raytrace.comp.glsl.spv` uses SPIR-V's binary format, and isn't easily human-readable.
    However, one can convert it to ASCII format using `spirv-dis` in the Vulkan SDK,
    by running
    ~~~
    spirv-dis raytrace.comp.glsl.spv
    ~~~
    
    This will print out something like this:
    ~~~ output
    ; SPIR-V
    ; Version: 1.5
    ; Generator: Khronos Glslang Reference Front End; 10
    ; Bound: 16
    ; Schema: 0
                   OpCapability Shader
                   OpExtension "SPV_KHR_non_semantic_info"
              %2 = OpExtInstImport "GLSL.std.450"
              %8 = OpExtInstImport "NonSemantic.DebugPrintf"
                   OpMemoryModel Logical GLSL450
                   OpEntryPoint GLCompute %main "main"
                   OpExecutionMode %main LocalSize 16 8 1
              %1 = OpString "C:/(omitted)/shaders/raytrace.comp.glsl"
              %7 = OpString "Hello world!"
                   OpSource GLSL 460 %1 "#version 460
    #extension GL_EXT_debug_printf : require

    layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;

    void main()
    {
      debugPrintfEXT(\"Hello world!\");
    }"
                   OpSourceExtension "GL_EXT_debug_printf"
                   OpName %main "main"
                   OpModuleProcessed "client vulkan100"
                   OpModuleProcessed "target-env spirv1.5"
                   OpModuleProcessed "target-env vulkan1.2"
                   OpModuleProcessed "entry-point main"
                   OpModuleProcessed "use-storage-buffer"
                   OpDecorate %gl_WorkGroupSize BuiltIn WorkgroupSize
           %void = OpTypeVoid
              %4 = OpTypeFunction %void
           %uint = OpTypeInt 32 0
         %v3uint = OpTypeVector %uint 3
        %uint_16 = OpConstant %uint 16
         %uint_8 = OpConstant %uint 8
         %uint_1 = OpConstant %uint 1
    %gl_WorkGroupSize = OpConstantComposite %v3uint %uint_16 %uint_8 %uint_1
           %main = OpFunction %void None %4
              %6 = OpLabel
                   OpLine %1 8 0
              %9 = OpExtInst %void %8 1 %7
                   OpReturn
                   OpFunctionEnd
    ~~~
    
    Reading SPIR-V is outside the scope of this tutorial, but you can find out more
    about SPIR-V at https://www.khronos.org/spir/. 

!!! Tip Things One Can Do with SPIR-V
    SPIR-V lets you use languages other than GLSL with Vulkan. For instance, 
    [DirectXShaderCompiler](https://github.com/microsoft/DirectXShaderCompiler) can compile HLSL to SPIR-V,
    and [Slang](https://github.com/shader-slang/slang) can compile Slang and HLSL to SPIR-V.
    
    Tools such as [SPIRV-Cross](https://github.com/KhronosGroup/SPIRV-Cross)
    can go in the other direction, converting SPIR-V to GLSL, HLSL, and MSL.
    
    [SPIRV-Tools](https://github.com/KhronosGroup/SPIRV-Tools) can transform
    SPIR-V code, performing some optimizations, minifying code, and much more.
    
    Tools such as [SMOL-V](https://github.com/aras-p/smol-v) can make SPIR-V
    smaller and more compressible.

## Including Additional Headers

Most of the rest of this chapter will focus on running a compute shader from Vulkan.
To do this, we'll need to create a few objects. Here's a map of the objects we'll
create; an arrow from object A to object B means that creating object A requires
creating object B first.

*****************************************************************************
*
* .--------------------------.
* | VkShaderModule           |
* | (raytrace.comp.glsl.spv) |
* '--------------------------'
*                ^
*                |
* .---------------------------------.
* | VkPipelineShaderStageCreateInfo |    .-------------------.
* | (tells Vulkan the entry point,  |    | VkPipelineLayout  |
* | main(), and that this is a      |    | (empty for now -  |
* | compute shader)                 |    | see next chapter) |
* '---------------------------------'    '-------------------'
*                                 ^        ^
*                                 |        |
*                              .--------------.
*                              | VkPipeline   |
*                              | (the compute |
*                              | pipeline)    |
*                              '--------------'
*
*****************************************************************************

!!! Tip
    Make sure you're using a debug build as described in Section [Configure and Generate the Project using CMake] in order to see validation layer messages.

To start off, include two additional headers: `nvh/fileoperations.hpp`, for
loading files, and `nvvk/shaders_vk.hpp`, for creating shader modules from SPIR-V.

Your entire include section should now look like this:

~~~ cpp
#define STB_IMAGE_WRITE_IMPLEMENTATION
#include <stb_image_write.h>

#include <nvh/fileoperations.hpp>  // For nvh::loadFile
#include <nvvk/context_vk.hpp>
#include <nvvk/error_vk.hpp>              // For NVVK_CHECK
#include <nvvk/resourceallocator_vk.hpp>  // For NVVK memory allocators
#include <nvvk/shaders_vk.hpp>            // For nvvk::createShaderModule
~~~

## Defining the Work Group Width and Height

Add the following constants after `render_width` and `render_height`:

~~~ cpp
static const uint32_t workgroup_width  = 16;
static const uint32_t workgroup_height = 8;
~~~

## Enabling Debug Printf

To enable Debug Printf, we need to add the `VK_KHR_shader_non_semantic_info`
extension, then tell the validation layer to enable Debug Printf, and to print
Debug Printf's text to the standard output.

Don't worry if this seems complex -- we only use Debug Printf in this chapter, and we'll
remove this code in the next chapter. For
more information, please see https://github.com/KhronosGroup/Vulkan-ValidationLayers/blob/master/docs/debug_printf.md.

After adding the `VK_KHR_RAY_QUERY_EXTENSION_NAME` (`VK_KHR_ray_query`) extension, add the following code:

~~~ cpp
  // Add the required device extensions for Debug Printf. If this is confusing,
  // don't worry; we'll remove this in the next chapter.
  deviceInfo.addDeviceExtension(VK_KHR_SHADER_NON_SEMANTIC_INFO_EXTENSION_NAME);
  VkValidationFeatureEnableEXT validationFeatureToEnable = VK_VALIDATION_FEATURE_ENABLE_DEBUG_PRINTF_EXT;
  VkValidationFeaturesEXT      validationInfo{.sType = VK_STRUCTURE_TYPE_VALIDATION_FEATURES_EXT,
                                              .enabledValidationFeatureCount = 1,
                                              .pEnabledValidationFeatures    = &validationFeatureToEnable};
  deviceInfo.instanceCreateInfoExt = &validationInfo;
#ifdef _WIN32
  _putenv_s("DEBUG_PRINTF_TO_STDOUT", "1");
#else   // If not _WIN32
  static char putenvString[] = "DEBUG_PRINTF_TO_STDOUT=1";
  putenv(putenvString);
#endif  // _WIN32
~~~

## Creating a Shader Module

To create a **shader module**, we'll need to:
* Find the SPIR-V file `raytrace.comp.glsl.spv` (since the application might be launched from several different directories)
* Load it into memory
* Create a shader module from it

`nvh::loadFile` can help with the first two steps. First, define a `std::vector`
of directories to search for `raytrace.comp.glsl.spv` after creating `nvvk::Buffer buffer`:

~~~ cpp
  const std::string        exePath(argv[0], std::string(argv[0]).find_last_of("/\\") + 1);
  std::vector<std::string> searchPaths = {exePath + PROJECT_RELDIRECTORY, exePath + PROJECT_RELDIRECTORY "..",
                                          exePath + PROJECT_RELDIRECTORY "../..", exePath + PROJECT_NAME};
~~~

Then, after creating the command pool, find, load, and create a shader module as follows. `nvh::loadFile` finds and loads
the file into a `std::string`; `nvvk::createShaderModule` is a helper function for `vkCreateShaderModule`.

~~~ cpp
  // Shader loading and pipeline creation
  VkShaderModule rayTraceModule =
      nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.comp.glsl.spv", true, searchPaths));
~~~

!!! Tip Why create the search paths so early?
    We'll later use these search paths to load an OBJ file for the scene.

## Specifying the Shader Stage and Entry Point

Technically, a SPIR-V file could have multiple **entry points**: one compute pipeline could use `main()`, while another could use `main2()`. This is why Vulkan refers to it as a **shader module**.

To tell the compute pipeline which entry point to use (and that this is a compute shader), create a `VkPipelineShaderStageCreateInfo` object as follows:

~~~ cpp
  // Describes the entrypoint and the stage to use for this shader module in the pipeline
  VkPipelineShaderStageCreateInfo shaderStageCreateInfo{.sType  = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO,
                                                        .stage  = VK_SHADER_STAGE_COMPUTE_BIT,
                                                        .module = rayTraceModule,
                                                        .pName  = "main"};
~~~

## An Empty Pipeline Layout

In C++, you can pass arguments to functions. Keeping with the function analogy, **descriptors** are one way of passing data to shaders. However, we don't need to pass anything to this chapter's compute shader. We still need to create an empty **pipeline layout**, which is sort of like defining a C function

~~~ cpp
void func(void);
~~~

Here's how to create an empty pipeline layout. You don't have to worry about what this code means for now, since we'll replace it in the next chapter:

~~~ cpp
  // For the moment, create an empty pipeline layout. You can ignore this code
  // for now; we'll replace it in the next chapter.
  VkPipelineLayoutCreateInfo pipelineLayoutCreateInfo{.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO,  //
                                                      .setLayoutCount         = 0,                             //
                                                      .pushConstantRangeCount = 0};
  VkPipelineLayout pipelineLayout;
  NVVK_CHECK(vkCreatePipelineLayout(context, &pipelineLayoutCreateInfo, VK_NULL_HANDLE, &pipelineLayout));
~~~

## Creating the Compute Pipeline

Finally, we'll create the compute pipeline using `vkCreateComputePipelines` and a `VkComputePipelineCreateInfo` object.
`VkComputePipelineCreateInfo` looks like this:

~~~ cpp
typedef struct VkComputePipelineCreateInfo {
    VkStructureType                    sType;
    const void*                        pNext;
    VkPipelineCreateFlags              flags; // Lets one modify how the pipeline works
    VkPipelineShaderStageCreateInfo    stage; // Created above
    VkPipelineLayout                   layout; // Created above
    VkPipeline                         basePipelineHandle; // Used for deriving compute pipelines -
    int32_t                            basePipelineIndex;  // we won't use this in this tutorial.
} VkComputePipelineCreateInfo;
~~~

Add code to fill out this structure as follows:

~~~ cpp
  // Create the compute pipeline
  VkComputePipelineCreateInfo pipelineCreateInfo{.sType  = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO,  //
                                                 .stage  = shaderStageCreateInfo,                           //
                                                 .layout = pipelineLayout};
  // Don't modify flags, basePipelineHandle, or basePipelineIndex
~~~

Finally, create the compute pipeline as follows. It's possible to create multiple compute pipelines at once, but
here we only need to create one:

~~~ cpp
  VkPipeline computePipeline;
  NVVK_CHECK(vkCreateComputePipelines(context,                 // Device
                                      VK_NULL_HANDLE,          // Pipeline cache (uses default)
                                      1, &pipelineCreateInfo,  // Compute pipeline create info
                                      VK_NULL_HANDLE,          // Allocator (uses default)
                                      &computePipeline));      // Output
~~~

## Dispatching the Compute Pipeline

First, remove the following lines that set `fillValue` and `fillValueU32` and then call `vkCmdFillBuffer` -- we'll run
the compute shader instead!

~~~ cpp
  // Fill the buffer
  const float     fillValue    = 0.5f;
  const uint32_t& fillValueU32 = reinterpret_cast<const uint32_t&>(fillValue);
  vkCmdFillBuffer(cmdBuffer, buffer.buffer, 0, bufferSizeBytes, fillValueU32);
~~~

To dispatch a compute shader, we first need to bind the compute shader's pipeline (this is a state change operation):

~~~ cpp
  // Bind the compute shader pipeline
  vkCmdBindPipeline(cmdBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, computePipeline);
~~~

Finally, add an instruction to dispatch the currently bound compute pipeline. Here, we tell it that we only want to
run a 1 x 1 x 1 grid of workgroups:

~~~ cpp
  // Run the compute shader with one workgroup for now
  vkCmdDispatch(cmdBuffer, 1, 1, 1);
~~~

## Modifying the Pipeline Barrier

Since we changed from a transfer operation (`vkCmdFillBuffer`) to a shader, we need to change the pipeline barrier from a transfer-to-host pipeline barrier, to a shader-to-host pipeline barrier. Replace the pipeline barrier with the following:

~~~ cpp highlight
  // Add a command that says "Make it so that memory writes by the compute shader
~~~ cpp
  // are available to read from the CPU." (In other words, "Flush the GPU caches
  // so the CPU can read the data.") To do this, we use a memory barrier.
  VkMemoryBarrier memoryBarrier{.sType         = VK_STRUCTURE_TYPE_MEMORY_BARRIER,
~~~ cpp highlight
                                .srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT,  // Make shader writes
~~~ cpp
                                .dstAccessMask = VK_ACCESS_HOST_READ_BIT};    // Readable by the CPU
  vkCmdPipelineBarrier(cmdBuffer,                                             // The command buffer
~~~ cpp highlight
                       VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT,                  // From the compute shader
~~~ cpp
                       VK_PIPELINE_STAGE_HOST_BIT,                            // To the CPU
                       0,                                                     // No special flags
                       1, &memoryBarrier,                                     // An array of memory barriers
                       0, nullptr, 0, nullptr);                               // No other barriers
~~~

This pipeline barrier will remain the same throughout the rest of this page.

## Cleaning Up

Finally, make sure to destroy the objects we created in this chapter at the end of the program. Just before calling `vkFreeCommandBuffers`, add
the following lines:

~~~ cpp
  vkDestroyPipeline(context, computePipeline, nullptr);
  vkDestroyShaderModule(context, rayTraceModule, nullptr);
  vkDestroyPipelineLayout(context, pipelineLayout, nullptr);  // Will be removed in the next chapter
~~~

Now, you should be able to run the program! If all goes well, you'll see output like this:

~~~ output
INFO: UNASSIGNED-DEBUG-PRINTF
 --> Validation Information: [ UNASSIGNED-DEBUG-PRINTF ] Object 0: handle = 0x1cb865172d8, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x92394c89 | Hello world!
INFO: UNASSIGNED-DEBUG-PRINTF
 --> Validation Information: [ UNASSIGNED-DEBUG-PRINTF ] Object 0: handle = 0x1cb865172d8, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x92394c89 | Hello world!
...
WARNING: UNASSIGNED-DEBUG-PRINTF
 --> Validation Warning: [ UNASSIGNED-DEBUG-PRINTF ] Object 0: handle = 0x1cb865172d8, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x92394c89 | WARNING - Debug Printf message was truncated, likely due to a buffer size that was too small for the message
~~~

You can ignore this warning message, or read the box below to find out more about it.

!!! Tip How Debug Printf Works
    To fix the warning message, one can create a file named `vk_layer_settings.txt` in their executable's working directory, with the contents
    ~~~
    khronos_validation.printf_buffer_size = 8192
    ~~~
    
    Earlier, we mentioned that Vulkan layers can modify commands passing through them.
    When Debug Printf is enabled, the Validation Layer performs a significant amount of hidden modification.
    To work, it creates an extra buffer, intercepts pipeline layout creation
    to add an extra descriptor for this buffer, and intercepts the SPIR-V code to add instructions for `debugPrintfEXT`.
    When a shader instantiation calls `debugPrintfEXT`, it appends all `debugPrintfEXT`'s arguments,
    as well as some extra information, to this buffer (using atomic operations).
    When a shader that uses Debug Printf finishes, the Validation Layer reads this buffer back from GPU memory,
    formats all the format strings, and finally prints this information.
    
    The error that we saw earlier was due to this buffer (which has a default size of 1024 bytes) filling up
    (each `debugPrintfEXT` call uses more than 32 bytes, so since the shader called it from 128 threads and
    128 x 32 = 4096, this works out.)
    
    You can tell Debug Printf to print out some additional information by adding the line
    ~~~
    khronos_validation.printf_verbose = true
    ~~~
    to `vk_layer_settings.txt`. Running the program again will print out a series of lines like
    ~~~ output
    INFO: UNASSIGNED-DEBUG-PRINTF
     --> Validation Information: [ UNASSIGNED-DEBUG-PRINTF ] Object 0: handle = 0x1e5299ab778, name = queueGCT, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x92394c89 | Command buffer (0x1e535ccc958). Compute Index 0. Pipeline (0xa540ac0000000009). Shader Module (0x4c5b620000000007). Shader Instruction Index = 39.  Stage = Compute.  Global invocation ID (x, y, z) = (0, 0, 0 ) Hello from invocation (0, 0)!
     Debug shader printf message generated in file C:/(omitted)/shaders/raytrace.comp.glsl at line 8.Unable to find suitable #line directive in SPIR-V OpSource.
    ~~~
    
    Debug Printf is also a fairly new feature, so it might not work with all types of shaders.
    At the time of writing, the two main ways of debugging shaders are to write to storage buffers that
    are read back on the CPU, similar to the mechanism with `buffer` used here, or by visually debugging
    through special color outputs. A tool such as [Nsight Graphics](https://developer.nvidia.com/nsight-graphics)
    can visualize or inspect image and buffer content directly, removing some of the logistical debugging effort.
    For more information, please see the [Debug Printf documentation](https://github.com/KhronosGroup/Vulkan-ValidationLayers/blob/master/docs/debug_printf.md).

## Invocation Indices

Each invocation can look up where it is in the compute shader dispatch's grid by looking at OpenGL's `gl_GlobalInvocationID` variable.
To close out this chapter, we'll modify our GLSL code to print out these values.
To do this, change `raytrace.comp.glsl` to the following:

~~~ glsl
#version 460
#extension GL_EXT_debug_printf : require

layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;

void main()
{
  debugPrintfEXT("Hello from invocation (%d, %d)!\n", gl_GlobalInvocationID.x, gl_GlobalInvocationID.y);
}
~~~

Now run the sample again. You should see output that starts like this:

~~~ output
INFO: UNASSIGNED-DEBUG-PRINTF
 --> Validation Information: [ UNASSIGNED-DEBUG-PRINTF ] Object 0: handle = 0x283f9fc4c48, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x92394c89 | Hello from invocation (0, 0)!

INFO: UNASSIGNED-DEBUG-PRINTF
 --> Validation Information: [ UNASSIGNED-DEBUG-PRINTF ] Object 0: handle = 0x283f9fc4c48, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x92394c89 | Hello from invocation (1, 0)!

INFO: UNASSIGNED-DEBUG-PRINTF
 --> Validation Information: [ UNASSIGNED-DEBUG-PRINTF ] Object 0: handle = 0x283f9fc4c48, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x92394c89 | Hello from invocation (2, 0)!
~~~

All these invocations run in parallel, so the output may or may not be in order.

## Summary

In this chapter, we:
* Wrote and compiled a compute shader using GLSL;
* Loaded a shader module onto the GPU;
* Created a compute pipeline;
* Ran a compute shader using vkCmdDispatch;
* Used Debug Printf to print debug information from the GPU.

In the next chapter, we'll generate image instead of text data -- by adding a **descriptor** to write to the buffer from the compute shader!

<!-- Chapter -->
# Descriptors

In this chapter, we'll show how to program the GPU to write colors into `buffer`. To do this, we'll use a **descriptor** to pass a reference to `buffer` to the compute shader. At the end of this chapter, we'll have a multicolor image, like this:

![The output of the program at the end of the chapter.](images/6-descriptors.png)

The next chapter will show how to combine this with ray tracing.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/7_descriptors` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

In the previous chapter, we used the analogy that shaders are like functions a GPU calls. In this analogy, descriptors are like pointer arguments -- referencing data that the function can read or write.

However, descriptors are a bit more flexible than pointers. For instance, descriptors can reference subranges of resources -- like a range of data in a buffer. An application can bind sets of descriptors all at once, using **descriptor sets**. Descriptors take up memory, and so need to be allocated from **descriptor pools**.

A C++ function also has a function signature that describes the list of types of arguments it takes. For instance, the function
~~~ cpp
void functionName(BufferDescriptor array, ASDescriptor tlas, BufferDescriptor meshData);
~~~
has signature
~~~ cpp
void functionName(BufferDescriptor, ASDescriptor, BufferDescriptor);
~~~
Like function signatures, Vulkan pipelines specify the types of descriptors they use using **descriptor set layouts**. These can be combined with **push constant ranges** (another way of passing data to pipelines, which we'll describe in [the extras](extras.html#pushconstants)) to get a **pipeline layout** (which specifies all the resources the compute pipeline can use).
<!-- Technical note: Section 13.2.2 of the Vulkan spec says "Zero or more descriptor set layouts and zero or more push constant ranges are combined to form a pipeline layout object describing the complete set of resources that can be accessed by a pipeline." But removing "compute" here might be confusing for readers who are trying to think about graphics pipelines as well, since graphics pipelines can write to framebuffers specified in VkRenderPassBeginInfo. -->

In other words, there are five types of objects involved:
* descriptors (point to resource ranges),
* descriptor sets (contain sets of descriptors),
* descriptor pools (descriptor sets are allocated from these),
* descriptor set layouts (like a function signature, excluding push constant ranges), and
* pipeline layouts (like a complete function signature).

Here's an example of how this works. In this chapter, we'll only use a single **storage buffer descriptor** (there are [different types of descriptors](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkDescriptorType.html)) for the `VkBuffer` `buffer.buffer`. Later, we'll use descriptors to pass three storage buffers and one **acceleration structure** (described later) to the pipeline. In this latter case, here's what we would need to do:
1. Describe the "function signature" by creating a descriptor set layout using [vkCreateDescriptorSetLayout](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCreateDescriptorSetLayout.html): binding 0 would be a storage buffer descriptor, binding 1 would be an acceleration structure descriptor, and bindings 2 and 3 would also be storage buffer descriptors. (We choose this order while writing the GLSL code.)
2. Create a descriptor pool with space for three `VK_DESCRIPTOR_TYPE_STORAGE_BUFFER` and one `VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR` descriptors.
3. Allocate one descriptor set that matches that layout from the descriptor pool using [vkAllocateDescriptorSets](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkAllocateDescriptorSets.html).
4. Write the values for the descriptors in the descriptor set using [vkUpdateDescriptorSets](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkUpdateDescriptorSets.html).
5. Create a pipeline layout from the descriptor set layout using [vkCreatePipelineLayout](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCreatePipelineLayout.html).
6. Use the pipeline layout when making the pipeline (like we did with an empty pipeline layout in the last chapter).
7. Finally, call `vkCmdBindDescriptorSets` with the descriptor set when recording the command buffer to make the [`vkCmdDispatch`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCmdDispatch.html) call use the descriptors!

************************************************************************************
*
* .---------------------------.  .------------------.
* | VkDescriptorSetLayout     |  | VkDescriptorPool |
* | (like a function          |  | (contains space  |
* | signature of descriptors) |  | for descriptors) |
* '----------+-------------+--'  '---------+--------'
*            |              \              |                Written using
*            v               v             v            vkUpdateDescriptorSets
* .---------------------.  .--------------------------.           |
* | VkPipelineLayout    |  | VkDescriptorSet          |           |
* | (Like a complete    |  | (Allocated set of        | <--------'
* | function signature) |  | descriptors with layout) |
* '---------------------'  '--------------------------'
*
************************************************************************************

However, because our setup of these five objects is not too complex (a descriptor pool containing space for exactly one set), we'll use `nvvk::DescriptorSetContainer` to make creating descriptor-related objects a bit shorter. Interested readers can try to use Vulkan's descriptor set creation functions to create descriptors in more detail.

********************************************************
*
*                  .--------------------.
*                  | Layout information |
*                  '---------+----------'
*                            |
*                            v
* .---------------------------------------------------.
* |           nvvk::DescriptorSetContainer            |
* |             (helper for descriptors)              |
* '----------+--------------------------+-------------'
*            |                          |
*        contains                    contains
*            |                          |
*            v                          v
* .---------------------.  .--------------------------.
* | VkPipelineLayout    |  | VkDescriptorSet          |
* | (Like a complete    |  | (Allocated set of        |
* | function signature) |  | descriptors with layout) |
* '---------------------'  '--------------------------'
*
********************************************************

Here's the new graph of objects used to construct the compute pipeline this chapter:

************************************************************************************
*
* .--------------------------.
* | VkShaderModule           |                .------------------------------.
* | (raytrace.comp.glsl.spv) |                | nvvk::DescriptorSetContainer |
* '--------------+-----------'                | (helper for descriptors)     |
*                |                            '----+--------------------+----'
*                v                                 |                    |
* .---------------------------------.          contains              contains
* | VkPipelineShaderStageCreateInfo |              |                    |
* | (tells Vulkan the entry point,  |              v                    v
* | main(), and that this is a      |    .------------------.  .------------------.
* | compute shader)                 |    | VkPipelineLayout |  | VkDescriptorSet  |
* '-------------------------------+-'    '-+----------------'  '------------------'
*                                 |        |
*                                 v        v
*                              .--------------.
*                              | VkPipeline   |
*                              | (the compute |
*                              | pipeline)    |
*                              '--------------'
*
************************************************************************************

!!! Tip
    Vulkan's model for "binding arguments to functions" is extremely flexible.
    One can use the same pipeline layout for multiple pipelines, and that
    descriptor sets (which are sets of pointers to arguments in this analogy) can be used
    across different pipelines.
    
    In fact, one can even **bind** (use) multiple descriptor
    sets at the same time -- for instance, a shader could use a storage buffer from set 0,
    and a uniform buffer from set 1.
    
    Vulkan 1.2 also added the ability to work with very large binding tables and physical
    device addresses. This allows applications to do things like referring to resources
    using table indices or pointers, and storing these references in buffers or passing
    them as push constants. (We discuss push constants in [the extras](extras.html#pushconstants).)

Let's talk about using descriptor sets in command buffers.
To bind descriptor sets to a **pipeline bind point** (as of this writing, there are only
graphics, compute, and ray tracing bind points), we add a `vkCmdBindDescriptorSets`
command to a command buffer. When called with `VK_PIPELINE_BIND_POINT_COMPUTE`, this
tells the command buffer "Use the descriptors in these descriptor sets for compute
pipelines, until I bind different descriptor sets to this pipeline bind point."

In summary, our command buffer calls in this chapter will look like this:
* Begin the command buffer;
* Bind the `computePipeline` pipeline to the compute pipeline bind point;
* Bind the descriptor set to the compute pipeline bind point;
* Dispatch the compute pipeline with enough workgroups for the entire image;
* End the command buffer.

Let's get started!

## A New Compute Shader

This chapter's compute shader will generate this image:

![](images/6-descriptors.png)

For each $(x, y)$ pixel in the 800 x 600 image, it writes the linear RGB color $(x/800, y/600, 0)$.

<!-- Note: I think for many readers looking at GLSL the first time, the following is probably too steep a learning curve. Perhaps this could be broken up into a simpler shader?-->
In case the following is too much at once, for more information about shader programming, [The Book of Shaders](https://thebookofshaders.com/)
by Patricio Gonzalez Vivo and Jen Lowe is an excellent, gentle introduction to GLSL programming.

Replace the contents of `raytrace.comp.glsl` with the code below. We'll explain what each line of
this does immediately afterwards:

~~~ glsl
#version 460

layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;

layout(binding = 0, set = 0) buffer storageBuffer
{
  float imageData[];
};

void main()
{
  // The resolution of the buffer, which in this case is a hardcoded vector
  // of 2 unsigned integers:
  const uvec2 resolution = uvec2(800, 600);

  // Get the coordinates of the pixel for this invocation:
  //
  // .-------.-> x
  // |       |
  // |       |
  // '-------'
  // v
  // y
  const uvec2 pixel = gl_GlobalInvocationID.xy;

  // If the pixel is outside of the image, don't do anything:
  if((pixel.x >= resolution.x) || (pixel.y >= resolution.y))
  {
    return;
  }

  // Create a vector of 3 floats with a different color per pixel.
  const vec3 pixelColor = vec3(float(pixel.x) / resolution.x,  // Red
                               float(pixel.y) / resolution.y,  // Green
                               0.0);                           // Blue
  // Get the index of this invocation in the buffer:
  uint linearIndex = resolution.x * pixel.y + pixel.x;
  // Write the color to the buffer.
  imageData[3 * linearIndex + 0] = pixelColor.r;
  imageData[3 * linearIndex + 1] = pixelColor.g;
  imageData[3 * linearIndex + 2] = pixelColor.b;
}
~~~

### A Storage Buffer

This says that this shader can use a **storage buffer**. It'll interpret the storage
buffer as an array of floats named `imageData`. It also says that the descriptor for
this storage buffer will be in binding 0 of descriptor set 0. We'll use this information
when creating the descriptor sets.

~~~ glsl
layout(binding = 0, set = 0) buffer storageBuffer
{
  float imageData[];
};
~~~

### Coordinates

GLSL acts like C for the most part, but it includes some new vector types! This shader
uses `uvec2`, which is a vector of two unsigned ints, like `(100, 0)` or `(800, 600)`.
It also uses `vec3`, which is a vector of three floats, like `(0.1, 0.3, 0.2)` or `(-100.0, 3.14, 0.0)`.
One can add, subtract, multiply, and divide vectors, and much more. For a specification
of GLSL types, see [this page](http://learnwebgl.brown37.net/12_shader_language/glsl_data_types.html).

In this code, we create a `uvec2` with value `(800, 600)`:

~~~ glsl
  // The resolution of the buffer, which in this case is a hardcoded vector
  // of 2 unsigned integers:
  const uvec2 resolution = uvec2(800, 600);
~~~

(We'll show how to change the resolution without changing the GLSL code in [the extras](extras.html#pushconstants).)

GLSL compute shaders also have several built-in variables. `gl_GlobalInvocationID` is a `uvec3`
(vector of three unsigned ints) that contains the index of each invocation in the entire dispatch.
(In this case, this is the index of the pixel in the image.)

We use the `.xy` subscript to create a `uvec2` with its first and second coordinates.

~~~ glsl
  // Get the coordinates of the pixel for this invocation:
  //
  // .-------.-> x
  // |       |
  // |       |
  // '-------'
  // v
  // y
  const uvec2 pixel = gl_GlobalInvocationID.xy;
~~~

Here's some different methods for how one can get each component of a vector `vec` in GLSL:

Method          | First  | Second | Third  | Fourth
----------------|--------|--------|--------|-------
Square Brackets | vec[0] | vec[1] | vec[2] | vec[3]
Dotted XYZW     | vec.x  | vec.y  | vec.z  | vec.w
Dotted RGBA     | vec.r  | vec.g  | vec.b  | vec.a
Dotted STPQ     | vec.s  | vec.t  | vec.p  | vec.q

### Rejecting Invalid Accesses

Next, we make sure that we only consider pixels inside the image.
This works the same as C++:

<!-- Note: This won't ever happen because of the workgroup size dividing the resolution. Does it make sense to remove this until the arbitrary-resolution change? -->
~~~ glsl
  // If the pixel is outside of the image, don't do anything:
  if((pixel.x >= resolution.x) || (pixel.y >= resolution.y))
  {
    return;
  }
~~~

### Creating a Color

We then construct a `vec3` representing a color from the pixel
coordinates and resolution:

~~~ glsl
  // Create a vector of 3 floats with a different color per pixel.
  const vec3 pixelColor = vec3(float(pixel.x) / resolution.x,  // Red
                               float(pixel.y) / resolution.y,  // Green
                               0.0);                           // Blue
~~~

### Writing to a Buffer

Writing into a buffer works the same way as writing into an array in
C++:

~~~ glsl
  // Get the index of this invocation in the buffer:
  uint linearIndex = resolution.x * pixel.y + pixel.x;
  // Write the color to the buffer.
  imageData[3 * linearIndex + 0] = pixelColor.r;
  imageData[3 * linearIndex + 1] = pixelColor.g;
  imageData[3 * linearIndex + 2] = pixelColor.b;
~~~

Here, we write the red, then green, then blue color consecutively into the
buffer. (A later version will perform this more smoothly.)

## Including Additional Headers

We'll now write the code that binds the buffer to the compute pipeline.

To start out, include the `nvvk/descriptorsets_vk.hpp` header to
use `nvvk::DescriptorSetContainer`, in the include block near the start
of `main.cpp`:

~~~ cpp
#include <nvvk/descriptorsets_vk.hpp>  // For nvvk::DescriptorSetContainer
~~~

## Removing Debug Printf Code

We won't use Debug Printf for the remainder of the tutorial, so
*delete* the following code from the start of `main`:
~~~ cpp
  // Add the required device extensions for Debug Printf. If this is confusing,
  // don't worry; we'll remove this in the next chapter.
  deviceInfo.addDeviceExtension(VK_KHR_SHADER_NON_SEMANTIC_INFO_EXTENSION_NAME);
  VkValidationFeatureEnableEXT validationFeatureToEnable = VK_VALIDATION_FEATURE_ENABLE_DEBUG_PRINTF_EXT;
  VkValidationFeaturesEXT      validationInfo{.sType = VK_STRUCTURE_TYPE_VALIDATION_FEATURES_EXT,
                                              .enabledValidationFeatureCount = 1,
                                              .pEnabledValidationFeatures    = &validationFeatureToEnable};
  deviceInfo.instanceCreateInfoExt = &validationInfo;
#ifdef _WIN32
  _putenv_s("DEBUG_PRINTF_TO_STDOUT", "1");
#else  // If not _WIN32
  static char putenvString[] = "DEBUG_PRINTF_TO_STDOUT=1";
  putenv(putenvString);
#endif  // _WIN32
~~~

## Listing and Allocating Descriptors

To use `nvvk::DescriptorSetContainer`, we list the bindings we want to use, and then make
the `DescriptorSetContainer` create the corresponding descriptor objects.

In `raytrace.comp.glsl`, we listed a single resource at binding 0 of set 0. Like how
C++ has different ways of defining how functions can access arguments -- e.g. by value,
reference, constant reference, and so on, Vulkan has many different descriptor types.
Here, we use one **storage buffer descriptor** -- for a buffer that the shader can read
and write.
One can find the full list of descriptor types [here](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkDescriptorType.html).

Add the following code immediately after calling `vkCreateCommandPool`. This initializes
an `nvvk::DescriptorSetContainer`, tells it about the binding, and also tells it that
this descriptor can be accessed from a compute shader:

~~~ cpp
  // Here's the list of bindings for the descriptor set layout, from raytrace.comp.glsl:
  // 0 - a storage buffer (the buffer `buffer`)
  // That's it for now!
  nvvk::DescriptorSetContainer descriptorSetContainer(context);
  descriptorSetContainer.addBinding(0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~

The list of bindings forms a **descriptor set layout**. This is like the signature of a function,
but only listing pointer arguments. (A **pipeline layout** is like the signature of a function
listing all its arguments; the difference is that pipeline layouts can also include **push constants**,
another way of sending information to a pipeline described in [the extras](extras.html#images))

~~~ cpp
  // Create a layout from the list of bindings
  descriptorSetContainer.initLayout();
~~~

Once we have the descriptor set layout, we create a pool with enough space for one descriptor
set of that layout, and allocate that set. Then, we finally create a pipeline layout
from the descriptor set layout (with no additional push constants):

~~~ cpp
  // Create a descriptor pool from the list of bindings with space for 1 set, and allocate that set
  descriptorSetContainer.initPool(1);
  // Create a simple pipeline layout from the descriptor set layout:
  descriptorSetContainer.initPipeLayout();
~~~

## Writing Descriptors
Now that we've created a descriptor, we need to make it point to the buffer. Since the
descriptor lives on the GPU, this means we need to tell Vulkan to transfer some data
to the GPU.

To do this,
we create a `VkDescriptorBufferInfo` object that describes what range of memory the
descriptor should point to (as mentioned, Vulkan's model is flexible here). We then
create a `VkWriteDescriptorSet` (which says "write `descriptorBufferInfo` to the
descriptor in set 0, binding 0").

~~~ cpp
  // Write a single descriptor in the descriptor set.
  VkDescriptorBufferInfo descriptorBufferInfo{.buffer = buffer.buffer,     // The VkBuffer object
                                              .range  = bufferSizeBytes};  // The length of memory to bind; offset is 0.
  VkWriteDescriptorSet writeDescriptor = descriptorSetContainer.makeWrite(0 /*set index*/, 0 /*binding*/, &descriptorBufferInfo);
~~~

Finally, we call `vkUpdateDescriptorSets` to write the descriptor. As usual, this function
can write (and copy) multiple descriptor sets at once, but we only have one to write
at the moment:
~~~ cpp
  vkUpdateDescriptorSets(context,              // The context
                         1, &writeDescriptor,  // An array of VkWriteDescriptorSet objects
                         0, nullptr);          // An array of VkCopyDescriptorSet objects (unused)
~~~

This function isn't an instruction recorded to a command buffer, so the driver will update
the descriptor sets immediately. Therefore, it's good to make sure the GPU isn't using the
descriptor sets when this function writes to them.

## The New Pipeline Layout

**Remove** this code from the previous chapter that created an empty pipeline layout:

~~~ cpp
  // For the moment, create an empty pipeline layout. You can ignore this code
  // for now; we'll replace it in the next chapter.
  VkPipelineLayoutCreateInfo pipelineLayoutCreateInfo{.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO,  //
                                                      .setLayoutCount         = 0,                             //
                                                      .pushConstantRangeCount = 0};
  VkPipelineLayout pipelineLayout;
  NVVK_CHECK(vkCreatePipelineLayout(context, &pipelineLayoutCreateInfo, VK_NULL_HANDLE, &pipelineLayout));
~~~

Then change the code that creates `pipelineCreateInfo` as follows.
This is the final arrow in our object graph.

~~~ cpp
  VkComputePipelineCreateInfo pipelineCreateInfo{.sType  = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO,
                                                 .stage  = shaderStageCreateInfo,
                                                 .layout = descriptorSetContainer.getPipeLayout()};
~~~

## Binding the Descriptor Set and Dispatching

Finally, we need to record a state change saying to use this descriptor set for subsequent compute dispatches on this command buffer. The Vulkan function for this is [`vkCmdBindDescriptorSets`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCmdBindDescriptorSets.html).
It binds multiple descriptor sets at once, but we'll call it with only one descriptor set.
Here's its function signature, with comments:

~~~ cpp
void vkCmdBindDescriptorSets(
    VkCommandBuffer        commandBuffer,      // The command buffer to record this command to.
                                               // Subsequent commands in it will use these bindings.
    VkPipelineBindPoint    pipelineBindPoint,  // Where to bind the descriptors (in this case, the compute pipeline bind point).
                                               // Some commands use bindings from different bind points (e.g. compute dispatches
                                               // use the compute pipeline bind point, draw calls use the graphics bind point).
    VkPipelineLayout       layout,             // The pipeline's pipeline layout (like a function signature, excluding push constant ranges)
    uint32_t               firstSet,           // (Set to 0) Number to add to the descriptor set index
    uint32_t               descriptorSetCount, // Number of descriptor sets in the `pDescriptorSets` array
    const VkDescriptorSet* pDescriptorSets,    // A pointer to an array of descriptor sets
    uint32_t               dynamicOffsetCount, // (Unused) Number of dynamic offsets in the next array.
    const uint32_t*        pDynamicOffsets);   // (Unused) Pointer to an array of dynamic offsets.
~~~

Add the following code after calling `vkCmdBindPipeline` to bind the descriptor set container's one descriptor set to the compute bind point:

~~~ cpp
  // Bind the descriptor set
  VkDescriptorSet descriptorSet = descriptorSetContainer.getSet(0);
  vkCmdBindDescriptorSets(cmdBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, descriptorSetContainer.getPipeLayout(), 0, 1,
                          &descriptorSet, 0, nullptr);
~~~

In the previous chapter, we ran the compute shader with a single workgroup. But now, we need enough workgroups to cover the entire image. In other words, we need

$$\text{ceil}\left(\frac{\text{render_width}}{\text{workgroup_width}}\right) \times \text{ceil}\left(\frac{\text{render_height}}{\text{workgroup_height}}\right) \times 1$$

work groups. We can compute this using integer math, if a and b are positive `int`s (ignoring floating-point roundoff):

~~~ cpp
ceil(float(a)/float(b)) == (a + b - 1)/b
~~~

So, replace the `vkCmdDispatch` call with

~~~ cpp
  // Run the compute shader with enough workgroups to cover the entire buffer:
  vkCmdDispatch(cmdBuffer, (uint32_t(render_width) + workgroup_width - 1) / workgroup_width,
                (uint32_t(render_height) + workgroup_height - 1) / workgroup_height, 1);
~~~

## Cleaning Up

Finally, replace the `vkDestroyPipelineLayout` call at the end of `main()` with

~~~ cpp
  descriptorSetContainer.deinit();
~~~

You should now be able to build and run the program and see the image change to this!

![](images/6-descriptors.png)

## Cleaner Buffer Writing using Scalar Block Layout

In the GLSL shader, we interpreted the buffer as an array of floats, and wrote each
channel individually. But what if we could interpret the buffer as an array of `vec3`
colors, and write all three channels in a single line? We can do this using the
`scalar` layout qualifier.

To do this, we need to first add the `GL_EXT_scalar_block_layout` extension
in `raytrace.comp.glsl`

~~~ glsl
#extension GL_EXT_scalar_block_layout : require
~~~

Then change the definition of `imageData` to add the `scalar` layout qualifier:

~~~ glsl
// The scalar layout qualifier here means to align types according to the alignment
// of their scalar components, instead of e.g. padding them to std140 rules.
layout(binding = 0, set = 0, scalar) buffer storageBuffer
{
  vec3 imageData[];
};
~~~

If we didn't use the `scalar` keyword here, then for historical reasons, GLSL would
interpret `imageData` as if each element were padded to fit at a 4-floating-point element boundary.
By using `scalar`, we tell GLSL that we packed the elements consecutively in memory.

Finally, change the three lines that write `imageData` to the following line:

~~~ cpp
  imageData[linearIndex] = pixelColor;
~~~

!!! Tip Creating Glitch Effects by Using Different Layouts
    It's often a good idea to use `scalar` whenever possible in new code.
    But it's possible to create interesting glitch effects by purposely
    mismatching data layouts. Here's what this chapter's image would look
    like on one GPU if we didn't use `scalar` here (this uses undefined behavior):
    
    ![](images/6-noScalar.png)
    
    Please note that this accesses out-of-bounds memory,
    because not using the `scalar` layout qualifier here causes the GLSL
    compiler to think that the `vec3[]` member uses a memory layout
    where each `vec3` is aligned at a four-`float` boundary instead of
    every three `floats`. Normally, accessing out-of-bounds memory is
    undefined behavior, and can even result in ["device lost"](https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#devsandqueues-lost-device) errors,
    which are bad (they require recreating the `VkDevice` at least).
    However, if [VkPhysicalDeviceFeatures::robustBufferAccess](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkPhysicalDeviceFeatures.html)
    was enabled (`nvvk::Context` turns this off),
    then out-of-bounds accesses can't terminate the application, and
    instead follow [a set of rules](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkPhysicalDeviceFeatures.html) with some choices made by the driver.

## Summary

In this chapter, we've seen how one can use descriptors in descriptor sets to make
shaders read and write GPU memory. With the tools that we've covered already, one
can implement an enormous range of graphical techniques on the GPU. Shaders could
read and write each others' results, or one could run simulations by storing a
sequence of objects in a buffer and iteratively running a series of shaders.

In the next chapter, we'll create and pass an acceleration structure in a descriptor
to perform ray tracing.

!!! Tip Further Reading
    Christoph Kubisch gives more information about different ways to bind
    resources in his article [Vulkan Shader Resource Binding](https://developer.nvidia.com/vulkan-shader-resource-binding).

<!-- Chapter 7 -->
# Acceleration Structures and Ray Tracing

In this chapter, we'll modify the C++ and GLSL code to ray trace a mesh.
We'll show how to:
* Use [tinyobjloader](https://github.com/tinyobjloader/tinyobjloader) to load a mesh from an OBJ file;
* Create an acceleration structure (AS) for fast ray-scene intersection;
* Add a descriptor for the acceleration structure to the descriptor set;
* Modify the compute shader to use **ray queries** and ray trace a mesh.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/8_ray_tracing` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Including Additional Headers

This chapter uses three additional headers. We use C++'s `<array>` header for `std::array`,
which will help with the array of descriptor writes later in Section [Adding the TLAS to the Descriptor Set].

We also use syoyo's [tinyobjloader](https://github.com/tinyobjloader/tinyobjloader)
to load a mesh from an OBJ file. Much like `stb_image_write`, `tinyobjloader` requires
us to define `TINYOBJLOADER_IMPLEMENTATION` in one file (`main.cpp`) before including it.

Finally, the main section of this tutorial will also use the `nvvk::RaytracingBuilderKHR` object
to help with building common types of acceleration structures.
The NVIDIA Vulkan Ray Tracing Tutorial shows the implementation of these helpers [here](https://nvpro-samples.github.io/vk_raytracing_tutorial_KHR/vkrt_tutorial.md#accelerationstructure/bottom-levelaccelerationstructure/helperdetails:raytracingbuilder::buildblas()).

Change the block of includes at the top of `main.cpp` to the following -- we've highlighted
the lines of code to add:

~~~ cpp highlight
#include <array>
#include <cassert>
~~~ cpp
#define STB_IMAGE_WRITE_IMPLEMENTATION
#include <stb_image_write.h>
~~~ cpp highlight
#define TINYOBJLOADER_IMPLEMENTATION
#include <tiny_obj_loader.h>
~~~ cpp

#include <nvh/fileoperations.hpp>  // For nvh::loadFile
#include <nvvk/context_vk.hpp>
#include <nvvk/descriptorsets_vk.hpp>     // For nvvk::DescriptorSetContainer
#include <nvvk/error_vk.hpp>              // For NVVK_CHECK
~~~ cpp highlight
#include <nvvk/raytraceKHR_vk.hpp>        // For nvvk::RaytracingBuilderKHR
~~~ cpp
#include <nvvk/resourceallocator_vk.hpp>  // For NVVK memory allocators
#include <nvvk/shaders_vk.hpp>            // For nvvk::createShaderModule
~~~

## Refactoring Command Buffers

In this section, we'll use a second command buffer to upload vertex and index data to the GPU.
To make creating, using, and freeing a second command buffer, easier, let's take some of the
existing command buffer code and move it into its own function.

First, let's create a function to allocate a command buffer from a command pool, then begin recording it
(telling Vulkan that we'll only use the command buffer once). Add the following code before
`main()`:

~~~ cpp
VkCommandBuffer AllocateAndBeginOneTimeCommandBuffer(VkDevice device, VkCommandPool cmdPool)
{
  VkCommandBufferAllocateInfo cmdAllocInfo{.sType              = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,
                                           .commandPool        = cmdPool,
                                           .level              = VK_COMMAND_BUFFER_LEVEL_PRIMARY,
                                           .commandBufferCount = 1};
  VkCommandBuffer             cmdBuffer;
  NVVK_CHECK(vkAllocateCommandBuffers(device, &cmdAllocInfo, &cmdBuffer));
  VkCommandBufferBeginInfo beginInfo{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
                                     .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
  NVVK_CHECK(vkBeginCommandBuffer(cmdBuffer, &beginInfo));
  return cmdBuffer;
}
~~~

Now, inside `main()`, delete this code:

~~~ cpp
  // Allocate a command buffer
  VkCommandBufferAllocateInfo cmdAllocInfo{.sType              = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,
                                           .commandPool        = cmdPool,
                                           .level              = VK_COMMAND_BUFFER_LEVEL_PRIMARY,
                                           .commandBufferCount = 1};
  VkCommandBuffer             cmdBuffer;
  NVVK_CHECK(vkAllocateCommandBuffers(context, &cmdAllocInfo, &cmdBuffer));

  // Begin recording
  VkCommandBufferBeginInfo beginInfo{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
                                     .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
  NVVK_CHECK(vkBeginCommandBuffer(cmdBuffer, &beginInfo));
~~~

and replace it with the following call to `AllocateAndBeginOneTimeCommandBuffer`:

~~~ cpp
  // Create and start recording a command buffer
  VkCommandBuffer cmdBuffer = AllocateAndBeginOneTimeCommandBuffer(context, cmdPool);
~~~

Let's also create a function that ends recording a command buffer, then submits it
to a queue, waits for it to finish, and then frees the command buffer. Add the
following code before `main()`:

~~~ cpp
void EndSubmitWaitAndFreeCommandBuffer(VkDevice device, VkQueue queue, VkCommandPool cmdPool, VkCommandBuffer& cmdBuffer)
{
  NVVK_CHECK(vkEndCommandBuffer(cmdBuffer));
  VkSubmitInfo submitInfo{.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, .commandBufferCount = 1, .pCommandBuffers = &cmdBuffer};
  NVVK_CHECK(vkQueueSubmit(queue, 1, &submitInfo, VK_NULL_HANDLE));
  NVVK_CHECK(vkQueueWaitIdle(queue));
  vkFreeCommandBuffers(device, cmdPool, 1, &cmdBuffer);
}
~~~

Then, inside `main()`, delete this code:

~~~ cpp
  // End recording
  NVVK_CHECK(vkEndCommandBuffer(cmdBuffer));

  // Submit the command buffer
  VkSubmitInfo submitInfo{.sType              = VK_STRUCTURE_TYPE_SUBMIT_INFO,  //
                          .commandBufferCount = 1,                              //
                          .pCommandBuffers    = &cmdBuffer};
  NVVK_CHECK(vkQueueSubmit(context.m_queueGCT, 1, &submitInfo, VK_NULL_HANDLE));

  // Wait for the GPU to finish
  NVVK_CHECK(vkQueueWaitIdle(context.m_queueGCT));
~~~

and replace it with the following:

~~~ cpp
  // End and submit the command buffer, then wait for it to finish:
  EndSubmitWaitAndFreeCommandBuffer(context, context.m_queueGCT, cmdPool, cmdBuffer);
~~~

Also, delete the `vkFreeCommandBuffers` call at the end of `main()`:

~~~ cpp
  vkFreeCommandBuffers(context, cmdPool, 1, &cmdBuffer);
~~~

You should still be able to build and run the application.

!!! WARNING Command Buffer Techniques for Production Applications
    Currently, we wait for the GPU to finish after submitting each command buffer.
    While this uses only a line of code, in production applications, it's better to use more advanced
    synchronization techniques that let the CPU and GPU run in
    parallel.
    
    In our [Vulkan Dos and Don'ts](https://developer.nvidia.com/blog/vulkan-dos-donts/) article,
    we also recommend that real-time applications use around 15-30 command buffers (built on
    parallel CPU threads and 5-10 `vkQueueSubmit` calls per frame
    (remember that each `vkQueueSubmit` call can submit multiple command buffers).

Next, we'll show how to load a triangle mesh from an OBJ file.

## Triangle Meshes

Most 3D objects are represented using **triangle meshes**. (Some other common object
representations in computer graphics -- all of which can be implemented in ray tracing frameworks -- 
are 3D volumes, procedural objects, spline-based surfaces, particles, lines, and curves.)

The OBJ file format represents meshes using an array of vertices (which are 3D points, but can also
have some other **attributes**, such as a color per vertex, that we won't use), and an array of sets
of three indices. Each set of three indices corresponds to three vertices, which represent a triangle.

![A wireframe of an octahedron with its six vertices labeled.](images/8-octahedron.png)

For instance, the three indices `{0, 1, 2}` represent the triangle connecting vertices 0, 1, and 2
in that order in the octahedron above.

An octahedron has 6 vertices and 8 faces. If we used three 4-byte `float`s for the X,
Y, and Z coordinates of each vertex, the array of vertices would use 4 x 3 x 6 = 72 bytes. Since
there are 8 faces, we would use 3 x 8 = 24 indices, which could be `unsigned int`s, so the array of
indices would use 4 x 24 = 96 bytes.

For instance, to represent this octahedron in the right-handed Y-up coordinate
system described below, we might use the following vertices array:

~~~ c++
{ 0.,  1.,  0., // Vertex 0
  0.,  0.,  1., // Vertex 1
  1.,  0.,  0., // Vertex 2
  0.,  0., -1., // Vertex 3
 -1.,  0.,  0., // Vertex 4
  0., -1.,  0.} // Vertex 5
~~~

and the following indices array:

~~~ c++
{0, 1, 2, // Indices of face 0
 0, 2, 3, // Indices of face 1
 0, 3, 4, // ...
 0, 4, 1, //
 5, 2, 1, //
 5, 3, 2, //
 5, 4, 3, //
 5, 1, 4} // Indices of face 7
~~~

However, there are many ways to represent a mesh. Some engines might use only 16 bits
per index (or less) for this octahedron. Others might use triangle strips or triangle fans, or
use three vertices instead of indices per triangle (which is not very memory-efficient). When
creating the acceleration structure, we'll tell Vulkan how the application represents each mesh
in memory.

## Loading an OBJ File

We'll now use `tinyobjloader` to load the OBJ file located at `vk_mini_path_tracer/media/scenes/CornellBox-Original-Merged.obj`.
This file contains a single triangle mesh, which is a single-color version of the famous [Cornell Box test model](https://www.graphics.cornell.edu/online/box/):

![A multicolor version of the Cornell Box test model. Geometry and colors from Morgan McGuire's [Computer Graphics Archive](https://casual-effects.com/data/), rendered in Blender using clipping for tone mapping.](images/8-cornellBox.png)

This file comes from Morgan McGuire's [Computer Graphics Archive](https://casual-effects.com/data/) and is
licensed CC BY 3.0.

First, let's create a `tinyobj::ObjReader` object, and load the file into memory. Replace the
line defining `searchPaths` with

~~~ cpp
  // Load the mesh of the first shape from an OBJ file
  const std::string        exePath(argv[0], std::string(argv[0]).find_last_of("/\\") + 1);
  std::vector<std::string> searchPaths = {exePath + PROJECT_RELDIRECTORY, exePath + PROJECT_RELDIRECTORY "..",
                                          exePath + PROJECT_RELDIRECTORY "../..", exePath + PROJECT_NAME};
  tinyobj::ObjReader       reader;  // Used to read an OBJ file
  reader.ParseFromFile(nvh::findFile("scenes/CornellBox-Original-Merged.obj", searchPaths));
  assert(reader.Valid());  // Make sure tinyobj was able to parse this file
~~~

Here's how to get all the vertices in the OBJ file. (If we wanted to read things such as vertex colors,
we could do it by getting other members of `reader.GetAttrib()`)

~~~ cpp
  const std::vector<tinyobj::real_t>   objVertices = reader.GetAttrib().GetVertices();
~~~

Next, we need to get the indices of the mesh.
OBJ files can contain multiple **shapes**, each of which include a mesh, a set of lines, and a set of points.
We get the list of shapes, make sure that the file only contains one shape (the mesh), and then
get that shape:

~~~ cpp
  const std::vector<tinyobj::shape_t>& objShapes   = reader.GetShapes();  // All shapes in the file
  assert(objShapes.size() == 1);                                          // Check that this file has only one shape
  const tinyobj::shape_t& objShape = objShapes[0];                        // Get the first shape
~~~

To get the mesh's indices, we loop over each `index_t` index in `objShape.mesh.indices`. One unusual
thing about the OBJ file format is that each `index_t` index can point to a different vertex index,
normal index, and texture coordinate index (more on normals later). We only need the vertex indices.

~~~ cpp
  // Get the indices of the vertices of the first mesh of `objShape` in `attrib.vertices`:
  std::vector<uint32_t> objIndices;
  objIndices.reserve(objShape.mesh.indices.size());
  for(const tinyobj::index_t& index : objShape.mesh.indices)
  {
    objIndices.push_back(index.vertex_index);
  }
~~~

We've now retrieved the vertices and indices of the mesh!

!!! WARNING Non-Triangular Faces in OBJ Files
    This isn't a complete OBJ file importer -- instead of only triangles, faces in OBJ files
    can be quadrilaterals, pentagons, or general polygons. Unfortunately, building a
    full OBJ file reader that would [triangulate polygons](https://en.wikipedia.org/wiki/Polygon_triangulation)
    is outside the scope of this tutorial.

## Uploading Data to the GPU

Next, let's send the vertex and index data to the GPU so that we can render it. Previously, we
created an empty buffer, wrote to it on the GPU, and then read it back on the CPU. This time,
we'll use an overload of `allocator.createBuffer` to create a buffer and upload CPU data to the GPU.

The overload of `allocator.createBuffer` that we'll use looks like this:

~~~ cpp
template <typename T>
nvvk::Buffer createBuffer(const VkCommandBuffer& cmdBuf, // A command buffer to add upload commands to
                          const std::vector<T>&  data_,  // The data to upload to the command buffer
                          VkBufferUsageFlags     usage_, // Specifies in advance how the buffer can be used
                          // By default, the final buffer exists in GPU-local memory, which means
                          // that the CPU can't access it. (We created `buffer` with memory properties
                          // that allowed us to map it to and from the CPU.)
                          VkMemoryPropertyFlags  memProps_ = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)
{
  return createBuffer(cmdBuf, sizeof(T) * data_.size(), data_.data(), usage_, memProps_);
}
~~~

To use this, we'll create a command buffer, record instructions to upload the two arrays, then end the
command buffer and wait for it to finish.

After the call to `vkCreateCommandPool`, add the following code to start a command buffer:

~~~ cpp
  // Upload the vertex and index buffers to the GPU.
  nvvk::Buffer vertexBuffer, indexBuffer;
  {
    // Start a command buffer for uploading the buffers
    VkCommandBuffer uploadCmdBuffer = AllocateAndBeginOneTimeCommandBuffer(context, cmdPool);
~~~

Then call `allocator.createBuffer` twice to create buffers and to record commands to create and upload the data. We also need to specify how
we'll use the data in advance. We'll get their **device addresses** (i.e. the addresses of their memory on the GPU), and we'll use them as storage buffers (like how we currently use `buffer`) in the next chapter,
and as inputs when building acceleration structures.

~~~ cpp
    // We get these buffers' device addresses, and use them as storage buffers and build inputs.
    const VkBufferUsageFlags usage = VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT
                                     | VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_BUILD_INPUT_READ_ONLY_BIT_KHR;
    vertexBuffer = allocator.createBuffer(uploadCmdBuffer, objVertices, usage);
    indexBuffer  = allocator.createBuffer(uploadCmdBuffer, objIndices, usage);

~~~

Finally, end, submit, wait for, and free the command buffer. The allocator also allocates some temporary
**staging memory** to perform these uploads to GPU-local memory, so we tell the allocator to
free this memory here, after the command buffer finishes:

~~~ cpp
    EndSubmitWaitAndFreeCommandBuffer(context, context.m_queueGCT, cmdPool, uploadCmdBuffer);
    allocator.finalizeAndReleaseStaging();
  }
~~~

<!-- Custom admonition to handle diagrams -->
<div class ="admonition tip">
<div class="admonition-title">Internal Details: Allocator Staging Memory</div>
    The buffer we created only has the `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT` memory property, which means that
    the CPU can't access it. So to copy CPU memory to this buffer, the allocator uses temporary staging
    memory.
    
    In addition to the GPU-local buffer, the allocator creates a buffer of the same size with the
    `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT` memory properties (or reuses an existing buffer).
    Like `buffer`, this means that this staging memory is accessible by both the GPU and CPU. The allocator then
    maps the staging buffer's memory to the CPU, copies the CPU-side data to the staging buffer, and
    unmaps the staging buffer. Finally, it records a `vkCmdCopyBuffer` command to tell the GPU to
    copy data from the staging buffer to the GPU-local buffer.
    
    **************************************************************************
    * .------------------------------------.   .----------------------------.
    * |                RAM                 |   |            VRAM            |
    * |                                    |   |                            |
    * | .------.     .----------------.    |   |    .---------------------. |
    * | | Data | --> | Staging buffer | ---|---|--> | Device-local buffer | |
    * | '------'     '----------------'    |   |    '---------------------' |
    * |                                    |   |                            |
    * '------------------------------------'   '----------------------------'
    **************************************************************************
    
    For more information about GPU data transfers, please see the [`vk_async_resources`](https://github.com/nvpro-samples/vk_async_resources) sample.
</div>

## Acceleration Structures

When the GPU traces a ray, it finds places where a ray intersects the scene. We might want to find
the closest intersection (for things like opaque surfaces), all intersections (for things like
transmission), or the closest intersection that satisfies some condition (for scenes with
**alpha cutout textures**, where some parts of polygons are invisible).

Suppose for the moment that we only want to find where a ray intersects a mesh. One way to do
this would be to take every triangle in the mesh, and test if the ray intersects the triangle.
However, this would be inefficient -- a scene might have 50 million triangles, for
instance; if every pixel of a 3840 x 2160 image traced only one ray into the scene, a single
frame would require computing more than 414 trillion ray-triangle intersections!

One way to make this faster is to use an **acceleration structure**, such as a
[**bounding volume hierarchy**](https://en.wikipedia.org/wiki/Bounding_volume_hierarchy).
This groups triangles together into a tree of bounding boxes, based on their position in the mesh.
Now, when a ray traverses this acceleration structure, it must only consider a much
smaller set of potential intersections. (In computational complexity terms, for 
sufficiently nice cases, this reduces the average time to find a ray-mesh intersection
from $O(n)$ to $O(\log n)$, where $n$ is the number of triangles in the mesh.)

Additionally, RTX GPUs contain RT Cores, which are hardware units that implement
acceleration structure traversal and ray-triangle intersections. This makes
ray tracing significantly faster compared to similar GPUs without RT Cores.
As a result, the RTX 2080 Ti, for instance, [can compute more than 10 billion closest-hit
ray-mesh intersections per second on standard test models](https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf#page=38).

What we've described above is an acceleration structure for a single mesh. However,
scenes usually contain many meshes, including multiple instances of the same mesh
with different scales, rotations, and positions. To ray trace this quickly, Vulkan
can use an acceleration structure over **instances** of other acceleration structures!

![A diagram of an acceleration structure. There are three bottom-level acceleration structures (BLASes) of triangles, and a top-level acceleration structure containing four instances. Two instances point to the same BLAS; this could be used to include multiple copies of a model (perhaps with variations) at different places in a scene. From the [NVIDIA Vulkan Ray Tracing Tutorial](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR).](images/8-AccelerationStructure.svg)

More specifically, Vulkan ray tracing uses a two-level acceleration structure format.
**Bottom-level acceleration structures** (BLASes) are acceleration structures of
triangles (or of bounding boxes of procedural objects), and
**top-level acceleration structures** are acceleration structures of **instances**,
each of which point to a BLAS, include a **transform** (describing the position,
rotation, translation, and skew of the instance using a 3 x 4 affine transformation matrix),
and also include some additional info described below.

!!! Tip Instancing with TLASes
    It's possible to include multiple copies of the same mesh by making multiple instances
    point to the same BLAS, with different transforms. However, perhaps one might want
    to have multiple copies of the same mesh (i.e. they have the same shape), but that
    look different. 
    
    Luckily, instances include information that make this possible! Each instance can
    contain a 24-bit **instance ID** and a 24-bit **shader offset index**. Both of
    these can be accessed from ray queries. Shader offset indices have special meaning
    when looking up shaders in shader binding tables in ray tracing pipelines, not ray queries.

!!! Tip Examples for Instanced and Procedural Objects
    The only object in this tutorial is a mesh, but if you're interested in instancing
    and procedural objects (using intersection shaders), make sure to check out
    the NVIDIA Vulkan Ray Tracing Tutorial's Extra Tutorials at
    https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR.

To build an acceleration structure, we describe the objects (such as triangles or
instances), and then give this description to Vulkan -- in most cases, the GPU then
takes care of generating the tree structure. However, because there are so many
ways to specify geometry, there are many questions we must answer.

For instance, here are some of the things that need to be specified when creating a BLAS:
* Where are the vertex and index buffers on the GPU?
* How are the vertices laid out in memory? (In this case, they're sets of 3 floating-point numbers laid out consecutively.)
* What format do the indices use? (In this case, they're all unsigned integers.)
* How many vertices are there?
* Does the BLAS contain triangles or procedural objects (initially represented by axis-aligned bounding boxes)?
* How many triangles are there?
* What are the starting and ending triangles?
* Should the AS builder prioritize building the AS quickly, or prioritize structuring the AS such that invocations can trace rays faster?
* Should the AS be compacted? (This is a step that makes the AS use less memory.)
* Should the AS be updateable? (This makes it so that if a mesh changes shape over time, the AS can be quickly re-fit without changing the tree structure, instead of rebuilt. This sample only renders one frame, so we won't use it.)

When creating a TLAS, on the other hand, `nvvk::RaytracingBuilderKHR` can make use
of a list of `nvvk::RaytracingBuilderKHR::Instance` instance objects, which requires
less specification.

That said, creating the BLAS and TLAS doesn't take too long in the grand scheme of things
-- it's about 48 lines, which will wind up forming most of the rest of the program. Let's get started!

### Adding `GetBufferDeviceAddress`

First, let's add a function that gets the device address of a `VkBuffer`. As mentioned above,
a device address is like the address of a piece of memory on the GPU. To get the address
of a buffer, we create a `VkBufferDeviceAddressInfo` object specifying the `VkBuffer`,
then call `vkGetBufferDeviceAddress`.

Add the following function after `EndSubmitWaitAndFreeCommandBuffer`:

~~~ cpp
VkDeviceAddress GetBufferDeviceAddress(VkDevice device, VkBuffer buffer)
{
  VkBufferDeviceAddressInfo addressInfo{.sType = VK_STRUCTURE_TYPE_BUFFER_DEVICE_ADDRESS_INFO, .buffer = buffer};
  return vkGetBufferDeviceAddress(device, &addressInfo);
}
~~~

### Creating the BLAS

We create bottom-level acceleration structures by passing a vector of `nvvk::RaytracingBuilderKHR::BlasInput` objects,
the input information used to create the BLAS, to `nvvk::RaytracingBuilderKHR::buildBlas`.

To create a `nvvk::RaytracingBuilderKHR::BlasInput`, we'll need to create three objects:
1. `VkAccelerationStructureGeometryTrianglesDataKHR` specifies where to find the vertex and index buffers, as well as how they're laid out in memory (e.g. how we formatted the vertices and indexes), and the maximum index in the array of vertices we use.
2. `VkAccelerationStructureGeometryKHR` points to the first object, and adds **geometry flags** (in this case, saying that the geometry has no transparent surfaces).
3. `VkAccelerationStructureBuildRangeInfoKHR` specifies the range of data to use, including the number of triangles. It's possible to use this to tell the AS builder to, say, include only the first half of triangles, add a constant value to all indices, or look up a transformation matrix to apply from another buffer, but we'll use mostly-default settings.

#### `VkAccelerationStructureGeometryTrianglesDataKHR`

Add the following code after uploading the vertex and index buffers.

First, define the vector of `BlasInput` objects, start creating one, and then get the device addresses of the vertex and index buffers:

~~~ cpp
  // Describe the bottom-level acceleration structure (BLAS)
  std::vector<nvvk::RaytracingBuilderKHR::BlasInput> blases;
  {
    nvvk::RaytracingBuilderKHR::BlasInput blas;
    // Get the device addresses of the vertex and index buffers
    VkDeviceAddress vertexBufferAddress = GetBufferDeviceAddress(context, vertexBuffer.buffer);
    VkDeviceAddress indexBufferAddress  = GetBufferDeviceAddress(context, indexBuffer.buffer);
~~~

Then, describe the `VkAccelerationStructureGeometryTrianglesDataKHR` object.
This object says:
* The vertex format is `VK_FORMAT_R32G32B32_SFLOAT` (i.e. a vertex consists of three 32-bit floating-point coordinates in X, Y, Z order).
* The vertices can be found at `vertexBufferAddress`.
* Each vertex starts $3 * \text{sizeof}(\text{float}) = 12$ bytes after the next. (This is useful in case one has a buffer where e.g. there's a set of XYZ coordinates for a vertex, then an RGB color, then another set of XYZ coordinates, and so on.)
* We'll use vertices 0 through `objVertices.size()/3 - 1`. The division by 3 is because `objVertices` is a vector of floats, and each vertex uses 3 floats.
* The index format is `VK_INDEX_TYPE_UINT32` (i.e. an index is a 32-bit unsigned integer).
* The indices can be found at `indexBufferAddress`.
* There is no transformation matrix. (This can be a pointer to a matrix in device memory -- but here we use 0 to indicate that the builder should use the identity matrix, which doesn't affect the position of the vertices.)

~~~ cpp
    // Specify where the builder can find the vertices and indices for triangles, and their formats:
    VkAccelerationStructureGeometryTrianglesDataKHR triangles{
        .sType         = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_GEOMETRY_TRIANGLES_DATA_KHR,
        .vertexFormat  = VK_FORMAT_R32G32B32_SFLOAT,
        .vertexData    = {.deviceAddress = vertexBufferAddress},
        .vertexStride  = 3 * sizeof(float),
        .maxVertex     = static_cast<uint32_t>(objVertices.size() / 3 - 1),
        .indexType     = VK_INDEX_TYPE_UINT32,
        .indexData     = {.deviceAddress = indexBufferAddress},
        .transformData = {.deviceAddress = 0}  // No transform
    };
~~~

!!! Tip The `VkFormat` Enum
    `VK_FORMAT_R32G32B32_SFLOAT` specifies three floating-point values in order. It looks like a color format, but it can also be used as a vertex format. For more information, as well as for a way to find out which formats can be used as vertex formats, see [`vkGetPhysicalDeviceFormatProperties`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkGetPhysicalDeviceFormatProperties.html).

#### `VkAccelerationStructureGeometryKHR`

This object says:
* The geometry is a triangle mesh (as opposed to a list of instance bounding boxes or a procedural object).
* The object describing the triangle mesh is `triangles`.
* The geometry uses the `VK_GEOMETRY_OPAQUE_BIT_KHR` flag. (For a full list of flags, please see [https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGeometryFlagBitsNV.html](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGeometryFlagBitsNV.html).)

~~~ cpp
    // Create a VkAccelerationStructureGeometryKHR object that says it handles opaque triangles and points to the above:
    VkAccelerationStructureGeometryKHR geometry{.sType        = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_GEOMETRY_KHR,
                                                .geometryType = VK_GEOMETRY_TYPE_TRIANGLES_KHR,
                                                .geometry     = {.triangles = triangles},
                                                .flags        = VK_GEOMETRY_OPAQUE_BIT_KHR};
    blas.asGeometry.push_back(geometry);
~~~

#### `VkAccelerationStructureBuildOffsetInfoKHR`

As mentioned above, we don't use any of the features of this structure; the only field that is not zero is `primitiveCount`, which is the number of triangles in this case (since we said that this was a triangle mesh in `geometry.geometryType`).

~~~ cpp
    // Create offset info that allows us to say how many triangles and vertices to read
    VkAccelerationStructureBuildRangeInfoKHR offsetInfo{
        .primitiveCount  = static_cast<uint32_t>(objIndices.size() / 3),  // Number of triangles
        .primitiveOffset = 0,  // Offset added when looking up triangles
        .firstVertex     = 0,  // Offset added when looking up vertices in the vertex buffer
        .transformOffset = 0   // Offset added when looking up transformation matrices, if we used them
    };
    blas.asBuildOffsetInfo.push_back(offsetInfo);
    blases.push_back(blas);
  }
~~~

#### Running `buildBlas`

Finally, create the ray tracing builder, tell it about the Vulkan context, memory allocator, and queue to use, and then build the BLAS. Here, we tell it to prefer fast ray tracing performance, instead of preferring fast AS builds.

~~~ cpp
  // Create the BLAS
  nvvk::RaytracingBuilderKHR raytracingBuilder;
  raytracingBuilder.setup(context, &allocator, context.m_queueGCT);
  raytracingBuilder.buildBlas(blases, VK_BUILD_ACCELERATION_STRUCTURE_PREFER_FAST_TRACE_BIT_KHR);
~~~

This will now build a bottom-level acceleration structure!

!!! Tip `buildBlas` Runs Commands; Why Doesn't It Take a Command Buffer?
    `nvvk::RaytracingBuilderKHR` creates, records, ends, submits, and waits for command buffers,
    so that you don't have to write the code to do so for it.
    
    However, this also means that `nvvk::RaytracingBuilderKHR::buildBlas` makes
    the CPU thread wait until the GPU finishes building the BLAS. If one needs to update
    or rebuild a BLAS interactively, it's a good idea to adapt
    this function to run as asynchronously as possible.

!!! Tip Making Acceleration Structures Smaller Using Compaction
    By adding one flag bit here, it's possible to **compact** the BLAS, substantially
    reducing the amount of memory it uses. To see how to do this, check out
    [the extras](extras.html#compaction)!

### Creating the TLAS

As described above, a top-level acceleration structure (TLAS) is an acceleration
structure over an array of instances. These instances place BLASes within a
scene.

Each instance points to a BLAS and includes a
transformation matrix as well as several other properties that can change the
appearance of the BLAS. For instance, if we wanted to place several copies of an
object with different materials in the world (see the [Multiple Materials](extras.html#multiplematerials) section in the Extras),
we could create several instances where each instance points to the same BLAS,
but with different transform matrices and different shader indices.

We define each instance using a [VkAccelerationStructureInstanceKHR](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkAccelerationStructureInstanceKHR.html)
struct. This type has six fields:
* `uint64_t accelerationStructureReference` is the device address of the BLAS. This is a pointer to the start of the BLAS in device memory. We'll use `nvvk::RayTracingBuilderKHR::getBlasDeviceAddress(n)`, which gets the device address of BLAS *n*, but this is essentially a small wrapper around Vulkan's [vkGetAccelerationStructureDeviceAddressKHR](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkGetAccelerationStructureDeviceAddressKHR.html) function.
* `VkTransformMatrixKHR transform` is a 3 x 4 matrix transform that can be applied to translate, scale, rotate, and skew the instance. However, we use the identity matrix, which doesn't change vertex positions. See the "Affine Transformation Matrices" tip below for more information.
* `instanceCustomIndex` is a field we can use to store an arbitrary 24-bit number. When a ray query intersects a BLAS, we can access this number using `rayQueryGetIntersectionInstanceCustomIndexEXT`. We'll set it to 0 for now.
* `instanceShaderBindingTableRecordOffset` is interesting: it's designed to store an offset for looking up material shaders (changing how light interacts differently with different objects). This directly affects the control flow of ray tracing pipelines, which we'll cover in the Extras. However, we're using ray queries here, so this can also be an arbitrary 24-bit number for now. Like `instanceCustomIndex`, this is accessible using `rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT`. We'll also set it to 0 for now.
* `VkGeometryInstanceFlagsKHR flags` acts like `geometry.flags` above: it changes how rays intersect with the instance during ray traversal. For instance, if we didn't use `VK_GEOMETRY_OPAQUE_BIT_KHR` in `geometry.flags` above, we could have used `VK_GEOMETRY_INSTANCE_FORCE_OPAQUE_BIT_KHR`. The set of flags is technically slightly different, and is available at the [VkGeometryInstanceFlagBitsKHR page](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGeometryInstanceFlagBitsKHR.html). We'll use `VK_GEOMETRY_INSTANCE_TRIANGLE_FACING_CULL_DISABLE_BIT_KHR`, which turns off [backface culling](https://en.wikipedia.org/wiki/Back-face_culling) (where rays pass through triangles from one side). We enable this because the square shape at the top of the ceiling of the Cornell box represents a solid surface, but it's modeled as two triangles, rather than a full cuboid.
* `mask` is an 8-bit bitmask. We'll describe this in more detail in Section [GLSL: Ray Queries]. For now, know that it lets us programmatically select a set of instances to trace, without rebuilding the TLAS. We set it to 0xFF to include each instance in every ray trace.

Note that we only create one instance here. We'll see examples of multiple instances in [the Extras](extras.html#instancesandtransformationmatrices).

~~~ cpp
  // Create an instance pointing to this BLAS, and build it into a TLAS:
  std::vector<VkAccelerationStructureInstanceKHR> instances;
  {
    VkAccelerationStructureInstanceKHR instance{};
    instance.accelerationStructureReference = raytracingBuilder.getBlasDeviceAddress(0);  // The address of the BLAS in `blases` that this instance points to
    // Set the instance transform to the identity matrix:
    instance.transform.matrix[0][0] = instance.transform.matrix[1][1] = instance.transform.matrix[2][2] = 1.0f;
    instance.instanceCustomIndex = 0;  // 24 bits accessible to ray shaders via rayQueryGetIntersectionInstanceCustomIndexEXT
    // Used for a shader offset index, accessible via rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT
    instance.instanceShaderBindingTableRecordOffset = 0;
    instance.flags = VK_GEOMETRY_INSTANCE_TRIANGLE_FACING_CULL_DISABLE_BIT_KHR;  // How to trace this instance
    instance.mask  = 0xFF;
    instances.push_back(instance);
  }
~~~

Finally, launch the TLAS build by calling `nvvk::RaytracingBuilderKHR::buildTlas`:

~~~ cpp
  raytracingBuilder.buildTlas(instances, VK_BUILD_ACCELERATION_STRUCTURE_PREFER_FAST_TRACE_BIT_KHR);
~~~

Now all that's left on the C++ side is to add the TLAS to the descriptor set!

!!! Tip Affine Transformation Matrices
    Vulkan ray tracing uses 3 x 4 affine transformation matrices to transform meshes. These can be used to position, rotate, scale, and skew objects. (If you are familiar with 4 x 4 transformation matrices in computer graphics, these are the same, but with vectors multiplied on the right and the last row removed.)
    
    When using an `instance.transform` matrix $M$ to represent a transformed BLAS, the instance acts as if all the $v = (x, y, z)$ vertices have had the following transformation applied:
    
    $$\begin{aligned} M\mathbf{v} &= \begin{pmatrix}M[0][0] & M[0][1] & M[0][2] & M[0][3] \\ M[1][0] & M[1][1] & M[1][2] & M[1][3] \\ M[2][0] & M[2][1] & M[2][2] & M[2][3]\end{pmatrix} \begin{pmatrix}x\\y\\z\\1\end{pmatrix} \\
    &= \begin{pmatrix}M[0][0] & M[0][1] & M[0][2] \\ M[1][0] & M[1][1] & M[1][2] \\ M[2][0] & M[2][1] & M[2][2]\end{pmatrix} \begin{pmatrix}x\\y\\z\end{pmatrix} + \begin{pmatrix}M[0][3]\\M[1][3]\\M[2][3]\end{pmatrix} \\
    &= \begin{pmatrix}M[0][0]x + M[0][1]y + M[0][2]z + M[0][3]\\M[1][0]x + M[1][1]y + M[1][2]z + M[1][3]\\M[2][0]x + M[2][1]y + M[2][2]z + M[2][3]\end{pmatrix}\end{aligned}$$
    
    The second equation says that this is the sum of a 3 x 3 matrix-vector multiplication, plus a translation.
    
    If we use the 4 x 4 identity matrix with its fourth row removed,
    
    $$ M = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} $$
    
    then the vertices are unchanged:
    
    $$ Mv = \begin{pmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{pmatrix} \begin{pmatrix}x\\y\\z\end{pmatrix} + \begin{pmatrix}0\\0\\0\end{pmatrix} = \begin{pmatrix}x\\y\\z\end{pmatrix} $$
    
    For some examples of transformations, see [this chapter of the extras](extras.html#instancesandtransformationmatrices/transformationmatrices),
    and [this article](https://en.wikipedia.org/wiki/Transformation_matrix#Examples_in_2_dimensions).

## Adding the TLAS to the Descriptor Set

To trace rays from a shader, we need to add the acceleration structure to the descriptor set. To do this, let's make binding 1 an acceleration structure descriptor that can be accessed from the compute shader, by adding another line to the `addBinding` code:

~~~ cpp
  // Here's the list of bindings for the descriptor set layout, from raytrace.comp.glsl:
  // 0 - a storage buffer (the buffer `buffer`)
  // 1 - an acceleration structure (the TLAS)
  nvvk::DescriptorSetContainer descriptorSetContainer(context);
  descriptorSetContainer.addBinding(0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~ cpp highlight
  descriptorSetContainer.addBinding(1, VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~

We also must make this descriptor in the descriptor set point to the TLAS.  Find the following code block:

~~~ cpp
  // Write a single descriptor in the descriptor set.
  VkDescriptorBufferInfo descriptorBufferInfo{.buffer = buffer.buffer,    // The VkBuffer object
                                              .range  = bufferSizeBytes};  // The length of memory to bind; offset is 0.
  VkWriteDescriptorSet writeDescriptor = descriptorSetContainer.makeWrite(0 /*set index*/, 0 /*binding*/, &descriptorBufferInfo);
  vkUpdateDescriptorSets(context,              // The context
                         1, &writeDescriptor,  // An array of VkWriteDescriptorSet objects
                         0, nullptr);          // An array of VkCopyDescriptorSet objects (unused)
~~~

And replace it with this code:

~~~ cpp
  // Write values into the descriptor set.
  std::array<VkWriteDescriptorSet, 2> writeDescriptorSets;
  // 0
  VkDescriptorBufferInfo descriptorBufferInfo{.buffer = buffer.buffer,    // The VkBuffer object
                                              .range  = bufferSizeBytes};  // The length of memory to bind; offset is 0.
  writeDescriptorSets[0] = descriptorSetContainer.makeWrite(0 /*set index*/, 0 /*binding*/, &descriptorBufferInfo);
  // 1
  VkAccelerationStructureKHR tlasCopy = raytracingBuilder.getAccelerationStructure();  // So that we can take its address
  VkWriteDescriptorSetAccelerationStructureKHR descriptorAS{.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET_ACCELERATION_STRUCTURE_KHR,
                                                            .accelerationStructureCount = 1,
                                                            .pAccelerationStructures    = &tlasCopy};
  writeDescriptorSets[1] = descriptorSetContainer.makeWrite(0, 1, &descriptorAS);
  vkUpdateDescriptorSets(context,                                            // The context
                         static_cast<uint32_t>(writeDescriptorSets.size()),  // Number of VkWriteDescriptorSet objects
                         writeDescriptorSets.data(),                         // Pointer to VkWriteDescriptorSet objects
                         0, nullptr);  // An array of VkCopyDescriptorSet objects (unused)
~~~

This creates an array of `VkWriteDescriptorSet`s, instead of a single `VkWriteDescriptorSet` for passing to `vkUpdateDescriptorSets`.

!!! Tip What Is `std::array`?
    `std::array` is part of the C++ standard library. An `std::array` is a sized array:
    ~~~ cpp
    std::array<VkWriteDescriptorSet, 2> writeDescriptorSets;
    ~~~
    is like 
    ~~~ cpp
    VkWriteDescriptorSet writeDescriptorSets[2];
    ~~~
    We can get the number of elements in the `std::array` version using `writeDescriptorSets.size()`,
    while getting the size with the latter type usually uses a macro that can be complex for full generality (see [this question](https://stackoverflow.com/questions/4415524/common-array-length-macro-for-c)).
    Defining the array size in a single place reduces possible errors when we add more VkWriteDescriptorSet structs in
    Section [Adding Descriptors for the Vertex and Index Buffers].

!!! Tip `makeWrite` Overloads
    In the above code, we've called `descriptorSetContainer.makeWrite` first with a pointer to a `VkDescriptorBufferInfo`
    structure (for write 0), and then with a pointer to a `VkWriteDescriptorSetAccelerationStructureKHR` structure.
    
    A [`VkWriteDescriptorSet`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkWriteDescriptorSet.html) structure can describe writes to different descriptor types using its `descriptorType` member.
    This can switch between its `pImageInfo`, `pBufferInfo`, and `pTexelBufferView` members -- and in other cases,
    `vkUpdateDescriptorSets` looks for structures in its `pNext` linked list of structures.
    
    For instance, when we call `makeWrite` with a `VkDescriptorBufferInfo` object, it sets `pBufferInfo`, while
    when we call `makeWrite` with a `VkWriteDescriptorSetAccelerationStructureKHR` structure, it sets `pNext` to
    a pointer to that `VkWriteDescriptorSetAccelerationStructureNV` structure.
    
    `nvvk::DescriptorSetContainer` knows how to set the `descriptorType` field from the binding information we gave it
    when calling `addBinding`.

## Cleaning Up

Finally, we must destroy the ray tracing builder and the vertex and index buffers at the end of the program. Add these three lines after `descriptorSetContainer.deinit()`:

~~~ cpp
  raytracingBuilder.destroy();
  allocator.destroy(vertexBuffer);
  allocator.destroy(indexBuffer);
~~~

This concludes the C++ code for this chapter! We've now written almost all the C++ code for this tutorial; the next chapter will add the vertex and index buffers to the descriptor set, but then after that, we'll exclusively modify the shader code.

Let's now turn to modifying the compute shader to add a pinhole camera, and to trace rays into the scene. By the end of this chapter, we'll
be able to render the Cornell Box.

Now is also an excellent time to diff `main.cpp` against the the reference version of `main.cpp` in `8_ray_tracing`, and to check for any typos.

## GLSL: Adding the `GL_EXT_ray_query` Extension

To start out, this new compute shader will require the `GL_EXT_ray_query` extension, to trace rays. Add the following line of code to the block of `#extension` directives at the top of `raytrace.comp.glsl`:

~~~ glsl
#extension GL_EXT_ray_query : require
~~~

## GLSL: Accessing the TLAS

Next, let's make the new TLAS descriptor accessible from the compute shader. Unlike `imageData`, the TLAS is a single variable, not a storage buffer, so we use the `uniform` interface qualifier:

~~~ glsl
layout(binding = 1, set = 0) uniform accelerationStructureEXT tlas;
~~~

!!! Tip In Depth: Understanding GLSL's `buffer` and `uniform`
    So far, we've declared two things describing data that can pass in or out of the shader -- one named `storageBuffer`, and the other named `tlas`:
    
    ~~~ glsl
    layout(binding = 0, set = 0, scalar) buffer storageBuffer
    {
      vec3 imageData[];
    };
    layout(binding = 1, set = 0) uniform accelerationStructureEXT tlas;
    ~~~
    
    Here, `layout(binding = 0, set = 0, scalar)` and `layout(binding = 1, set = 0)` are **layout qualifiers**, and `buffer` and `uniform` are **storage qualifiers**, according to the GLSL grammar defined in the [GLSL specification][glslspec].
    
    Every variable in GLSL either has or does not have a storage qualifier. This describes how variables can be accessed. For instance, `tlas` has the `uniform` storage qualifier, and it corresponds to an acceleration structure, which is initialized through the API; shader invocations can read from but not write to it. A variable `vec3 rayOrigin` has no storage qualifier; shader invocations can read and write it, but the API can't access it. A variable `const uvec2 pixel` can't be changed once set. A `buffer` variable is stored in a buffer object, and can be both read and written. A variable with an `in` storage qualifier comes from another pipeline stage, and a variable with an `out` storage qualifier goes to another pipeline stage.
    For a list of storage qualifiers, see the Storage Qualifiers section of the [GLSL specification][glslspec].
    
    It's possible to have **interface blocks** such as `storageBuffer`, which are blocks of variables handled collectively, which all get a storage qualifier. For instance, the unsized array `imageData` is part of `storageBuffer`, and has the `buffer` storage qualifier (meaning that it is stored in a buffer object). These are specified in the "Interface Blocks" section of the [GLSL specification][glslspec].
    
    Variables corresponding to Vulkan descriptors -- whether storage buffers, uniform buffers, acceleration structures, or things like samplers, images, and textures -- will all have the `buffer` or `uniform` storage qualifiers.
    
    Here are four examples of how GLSL variable declarations map to Vulkan descriptors and push constants, described in the "Mapping from GLSL to SPIR-V" section of the [GLSL specification][glslspec]:
    
    * `tlas` corresponds to an acceleration structure descriptor in binding 1 of bound descriptor set 0. Since it's not a buffer, it gets the `uniform` storage qualifier.
    * `storageBuffer` corresponds to a storage buffer descriptor in binding 0 of bound descriptor set 0. The "buffer" storage qualifier on an interface block makes this a storage buffer, instead of a uniform buffer, described below.
    * If we had a declaration like this:
    
    ~~~ glsl
    layout(binding = 2, set = 0, scalar) uniform nameOfBlockHere
    {
      vec4   backgroundColor;
      mat4x4 viewMatrix;
      float  focusDistance;
      vec3   lightPositions[16];
      vec3   lightColors[16];
    };
    ~~~
    
    <!-- Visual hack to make the text appear to be indented -->
    <div style="margin-left:2.5em;">
    then this would correspond to a **uniform buffer** descriptor in binding 2 of bound descriptor set 0, because of the "uniform" storage qualifier on an interface block. The uniform buffer would contain five variables, two of which are sized arrays. (Uniform buffers are buffers that shader invocations can only read, not write. They also have a size limitation of about 65,536 bytes, while storage buffers don't have this limitation. Uniform buffer accesses also compile to different instructions than storage buffer accesses.) We don't use uniform buffers in this tutorial.
    </div>
    
    * If we had a declaration like this:
    
    ~~~ glsl
    layout(push_constant) uniform nameOfOtherBlockHere
    {
      uvec2 imageResolution;
    }
    ~~~
    
    <div style="margin-left:2.5em;">
    then this would correspond to the block of **push constants**. This is like a potentially faster uniform buffer -- but its values are recorded into the command buffer, and pushed to a relatively small pool of memory. We discuss push constants in [the extras](extras.html#pushconstants).
    </div>

[glslspec]: https://www.khronos.org/registry/OpenGL/specs/gl/GLSLangSpec.4.60.pdf

## GLSL: A Virtual Camera

Delete all the code in `main()` starting with the line that defines `const vec3 pixelColor`. (Don't delete the `if((pixel.x >= resolution.x) ...` check.) The rest of the GLSL code will, for each pixel:
* Define a virtual camera located at (-0.001, 1, 6);
* Find the direction of the camera ray passing through the center of the pixel;
* Initialize a ray query;
* Trace the ray until it finds the closest surface that the ray intersects;
* Get the **t-value** of the ray;
* Turn the t-value into a grayscale color;
* Write the grayscale color into a buffer. 

In Vulkan ray tracing, a **ray** is defined by its **origin** $\mathbf{o}$ and its **direction** $\mathbf{d}$. (These variables are bolded to signify that they're vectors.)

![A ray with origin o and direction d, with some points marked along it.](images/8-ray.svg)

A ray can also be expressed as a function that takes numbers and returns points:

$$ \mathbf{r}(t) = \mathbf{o} + t\mathbf{d} $$

Points along a ray are given by their values of $t$. For instance, $\mathbf{o}$ has a **t-value** of 0, $\mathbf{o} + \mathbf{d}$ has a t-value of 1, $\mathbf{o} + 1.5\mathbf{d}$ has a t-value of 1.5, and so on. The t-value is also the distance of a point along a ray from the origin of the ray, divided by the length of $\mathbf{d}$.

Here's a diagram of our scene. This scene uses the **OBJ coordinate system**, where the positive x axis points right, the positive y axis points up, and the positive Z axis points from the Cornell box towards the camera. This is a right-handed Y-up coordinate system.

![A diagram of the scene, showing the camera, its field of view, the Cornell box, and the positive X (red), Y (green), and Z (blue) axes of the coordinate system.](images/8-sceneDiagram.png)

We'll ray trace this scene using a virtual camera located at (-0.001, 1, 6). (In other words, we'll render the scene from that point of view.) This camera will be a pinhole camera, so all rays will have their origin at (-0.001, 1, 6), and each pixel will have a different direction.

To figure out the direction for each pixel, imagine placing a computer screen 1 unit in front of the camera origin, with a resolution of  resolution.x x resolution.y. Each ray will start at (-0.001, 1, 6), and will initially pass through the center of each pixel.
Let's define the **vertical field of view** of the camera by saying that the screen extends from $y=1-1/5$ to $y=1+1/5$ (this controls how large the Cornell Box appears).

Let's find the coordinates of a pixel on this screen. First, we'll transform the pixel coordinates
to look like this, where $a$ is the aspect ratio (width/height, in our case 800/600 = 4/3) of the screen:

****************************************
*
*         --------------> pixel.x
*  
*            screenUV.y
*                1
*    |    .------+------.
*    |    |      |      |
*    | -a +----- 0 -----+ a  screenUV.x
*    |    |      |      |
*    |    '------+------'
*    |          -1
*    v
* pixel.y
* 
****************************************

The center of the pixel is located at

$$(\text{pixel.x} + 0.5, \text{pixel.y} + 0.5)$$

Subtracting half the resolution gives coordinates in [-resolution.x/2, resolution.x/2] x [-resolution.y/2, resolution.y/2]:

$$\left(\text{pixel.x} + 0.5 - \frac{\text{resolution.x}}{2}, \text{pixel.y} + 0.5 - \frac{\text{resolution.y}}{2}\right).$$

We can multiply by $1/\text{resolution.y}$ to get coordinates in [-a/2, a/2] x [-1/2, 1/2], while keeping pixels square:

$$\left(\frac{\text{pixel.x} + 0.5 - \text{resolution.x}/2}{\text{resolution.y}}, \frac{\text{pixel.y} + 0.5 - \text{resolution.y}/2}{\text{resolution.y}}\right).$$

Finally, multiply the x coordinate by 2, and the y coordinate by -2 (to flip the y axis) to get the screen coordinates, in [-a, a] x [-1, 1]:

$$\textbf{screenUV} = \left(\frac{2 \text{pixel.x} + 1 - \text{resolution.x}}{\text{resolution.y}}, -\frac{2 \text{pixel.y} + 1 - \text{resolution.y}}{\text{resolution.y}}\right)$$

Here's how to get the ray direction from the screen UV and the vertical slope of the field of view:

$$\textbf{rayDirection} = (\text{fovVerticalSlope} * \text{screenUV.x}, \text{fovVerticalSlope} * \text{screenUV.y}, -1.0).$$

Add the following code to `main()` in `raytrace.comp.glsl` to define the camera origin and ray origin, and to compute the direction of the ray for the pixel:

``` glsl
  // This scene uses a right-handed coordinate system like the OBJ file format, where the
  // +x axis points right, the +y axis points up, and the -z axis points into the screen.
  // The camera is located at (-0.001, 1, 6).
  const vec3 cameraOrigin = vec3(-0.001, 1.0, 6.0);
  // Rays always originate at the camera for now. In the future, they'll
  // bounce around the scene.
  vec3 rayOrigin = cameraOrigin;
  // Compute the direction of the ray for this pixel. To do this, we first
  // transform the screen coordinates to look like this, where a is the
  // aspect ratio (width/height) of the screen:
  //           1
  //    .------+------.
  //    |      |      |
  // -a + ---- 0 ---- + a
  //    |      |      |
  //    '------+------'
  //          -1
  const vec2 screenUV = vec2(2.0 * (float(pixel.x) + 0.5 - 0.5 * resolution.x) / resolution.y,    //
                             -(2.0 * (float(pixel.y) + 0.5 - 0.5 * resolution.y) / resolution.y)  // Flip the y axis
  );
  // Next, define the field of view by the vertical slope of the topmost rays,
  // and create a ray direction:
  const float fovVerticalSlope = 1.0 / 5.0;
  vec3        rayDirection     = vec3(fovVerticalSlope * screenUV.x, fovVerticalSlope * screenUV.y, -1.0);
```

## GLSL: Ray Queries

Now, we'll cast a ray into the scene. To do this, we'll use a **ray query**, which answers the question "which parts of the scene intersect this ray"?

We'll first initialize a ray query. Then we'll repeatedly call `rayQueryProceedEXT` to loop over all the places where the ray intersects the geometry. The ray query automatically keeps track of the closest intersection, and when this loop finishes, we'll be able to find out information about this closest intersection (and if there was any intersection at all).

!!! Tip Where Can I Find Documentation For Ray Query Functions?
    Now, the best documentation for GLSL's ray query functions is the `GLSL_EXT_ray_query` extension specification itself, which one can find [here](https://github.com/KhronosGroup/GLSL/blob/master/extensions/ext/GLSL_EXT_ray_query.txt).

To initialize the ray query, we call `rayQueryInitializeEXT` with:
* The ray query object
* The top-level acceleration structure
* **Ray flags** that allow us to customize or speed up the search for intersections, and can override instance flags. We'll use the `gl_RayFlagsOpaqueEXT` flag, which means "there is no transparency". (See [GL_EXT_ray_tracing](https://github.com/KhronosGroup/GLSL/blob/master/extensions/ext/GLSL_EXT_ray_tracing.txt#L122-L130) for a full list)
* An 8-bit **ray mask**. (Instances have 8-bit masks; a ray can only intersect the instance if at least one bit is on in the same position in both the ray mask and the **instance mask**. This means that one can divide instances into different levels of detail, for instance, and tell rays to only trace particular levels of detail. We use a ray mask of 0xFF, and all instance masks are 0xFF. See the box below for more details.)
* The ray origin
* The minimum t-value for the ray, **tMin**
* The ray direction
* The maximum t-value for the ray, **tMax**

**tMin** and **tMax** let us search for intersections in a single line segment, the part of the ray where $ t_{\text{min}} \leq t \leq t_{\text{max}} $, instead of the entire ray.

!!! Tip Instance Masks Set to 0xFF
    The instance mask was part of `VkAccelerationStructureInstanceKHR`, which we created in Section [Creating the TLAS].
    Its default value (from `raytraceKHR_vk.hpp`) is 0xFF, which is why all instances have a mask of 0xFF in this tutorial.

Add this code to initialize the ray query object:

``` glsl
  // Trace the ray and see if and where it intersects the scene!
  // First, initialize a ray query object:
  rayQueryEXT rayQuery;
  rayQueryInitializeEXT(rayQuery,              // Ray query
                        tlas,                  // Top-level acceleration structure
                        gl_RayFlagsOpaqueEXT,  // Ray flags, here saying "treat all geometry as opaque"
                        0xFF,                  // 8-bit instance mask, here saying "trace against all instances"
                        rayOrigin,             // Ray origin
                        0.0,                   // Minimum t-value
                        rayDirection,          // Ray direction
                        10000.0);              // Maximum t-value
```

Next, we'll call `rayQueryProceedEXT` in a loop. Normally, every time we call this function, it proceeds to the next intersection, and updates the **committed intersection**, which is the closest (accepted) intersection so far. `rayQueryProceedEXT` returns `false` once we've iterated through all ray-scene intersections.

Add the following code:

``` glsl
  // Start traversal, and loop over all ray-scene intersections. When this finishes,
  // rayQuery stores a "committed" intersection, the closest intersection (if any).
  while(rayQueryProceedEXT(rayQuery))
  {
  }
```

Since we used the `gl_RayFlagsOpaqueEXT` flag, `rayQueryProceedEXT`
will immediately find the closest intersection, and then return `false`, without
executing any code within the block. Using a `while` loop here, though,
helps avoid errors -- it makes sure that the code would be correct even with
different flags, and we'll see how to use it creatively in the next chapter.

The ray cast has now finished, and `rayQuery` stores the closest intersection (the intersection with the lowest t-value)! Add this code to get the t-value of the committed intersection:

``` glsl
  // Get the t-value of the intersection (if there's no intersection, this will
  // be tMax = 10000.0). "true" says "get the committed intersection."
  const float t = rayQueryGetIntersectionTEXT(rayQuery, true);
```

## GLSL: Depth Mapping

Here, `t` is the depth of the scene along the ray. Let's divide it by 10 to map depths between 0 and 10 to the range from 0 to 1, then store it as a color:

~~~ glsl
  // Get the index of this invocation in the buffer:
  uint linearIndex       = resolution.x * pixel.y + pixel.x;
  // Give the pixel the color (t/10, t/10, t/10):
  imageData[linearIndex] = vec3(t / 10.0);
~~~

You should now be able to build and run the program, and see that it renders a depth map of the Cornell box to `out.hdr` using ray tracing! Darker values are closer, and brighter values are further away.

If the application fails to build or renders incorrectly, diff your code against `main.cpp` and `raytrace.comp.glsl` to check for any typos.

![A ray traced depth map of the scene. Darker values are closer, and brighter values are further away.](images/7-depthMap.png)

## Summary

Now that we've reached the end of this chapter, we've finally built an application that uses Vulkan ray tracing to render a depth map of a 3D model. We've completed almost all the C++ code, which at this point should be about 274 lines long.

To go from a buffer-only compute shader to a ray tracer in this chapter, we:
* Loaded a mesh from an OBJ file using `tinyobjloader`;
* Uploaded that mesh to the GPU;
* Created a BLAS and a TLAS using `nvvk::RaytracingBuilderKHR`;
* Added the TLAS to the descriptor set; and
* Rewrote the compute shader to use `rayQueryInitializeEXT`, `rayQueryProceedEXT`, and `rayQueryGetIntersectionTEXT` to do ray casting.

In the coming chapters, we'll see how to modify primarily the GLSL code to turn this from a depth map renderer to a path tracer.

<!-- Chapter 8 -->
# Four Uses of Intersection Data

To make a path tracer, we'll need to compute how light interacts with each part of the mesh, which means we'll need some more information about intersections other than t-values. This chapter explores ray queries, and includes four visualizations of the data one can get with them.

(Note: It is possible to build a path tracer with only the functions we've described so far! However, it would probably not be very efficient, and would be limited in many ways.)
<!-- Specifically, one could use additional ray queries to estimate the normal of the intersected polygon, potentially with numerical issues.-->

!!! Tip
    You can find the reference code for the last section of this chapter in the `checkpoints/9_intersection` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Counting Intersections with `rayQueryProceedEXT`

!!! TIP
    This section is intended to help the reader learn about the `rayQueryProceedEXT` while loop; they can skip to the Hits and Misses section if they want.

`rayQueryProceedEXT` iterates over ray-scene intersections. Now, the body of the `while` loop is empty, but we can use it to count how many times the ray intersects the scene.

The ray flags currently include `gl_RayFlagsOpaqueEXT`, which will make `rayQueryProceedEXT` immediately return the closest intersection. However, we want it to iterate over every intersection. Only removing the `gl_RayFlagsOpaqueEXT` flag won't work, since the instance has the equivalent `VK_GEOMETRY_OPAQUE_BIT_KHR` flag set. So, we need to override the instance setting by using the `gl_RayFlagsNoOpaqueEXT` ray flag. Replace the `rayQueryInitializeEXT` call with the following:

~~~ glsl
  rayQueryInitializeEXT(rayQuery,                // Ray query
                        tlas,                    // Top-level acceleration structure
~~~ glsl highlight
                        gl_RayFlagsNoOpaqueEXT,  // Ray flags, force iterating over every intersection (note: slow)
~~~ glsl
                        0xFF,                    // 8-bit instance mask, here saying "trace against all instances"
                        rayOrigin,               // Ray origin
                        0.0,                     // Minimum t-value
                        rayDirection,            // Ray direction
                        10000.0);                // Maximum t-value
~~~

Then replace the code after `rayQueryInitializeEXT` with the following:

~~~ glsl
  // Start traversal, and loop over all ray-scene intersections. When this finishes,
  // rayQuery stores a "committed" intersection, the closest intersection (if any).
~~~ glsl highlight
  float numIntersections = 0.0;
~~~ glsl
  while(rayQueryProceedEXT(rayQuery))
  {
~~~ glsl highlight
    numIntersections += 1.0;
~~~ glsl
  }

  // Get the index of this invocation in the buffer:
  uint linearIndex       = resolution.x * pixel.y + pixel.x;
~~~ glsl highlight
  imageData[linearIndex] = vec3(numIntersections / 10.0);
~~~

This counts the number of intersections (`numIntersections`) instead of using the following code (that it replaced) to get the distance to the first intersection:

~~~ glsl
const float t = rayQueryGetIntersectionTEXT(rayQuery, true);
~~~

Here's what `out.hdr` will look like now:

![](images/8-countingIntersections.png)

Brighter areas correspond to pixels with more ray-scene intersections.
Note that some pixels have more than two registered intersections, even though the
ray only intersects the scene once! That's because by default, the specification
doesn't guarantee that `rayQueryProceedEXT` runs only once per intersection.
To prevent this from happening, one way is to add the `VK_GEOMETRY_NO_DUPLICATE_ANY_HIT_INVOCATION_BIT_KHR`
flag to the `VkAccelerationStructureGeometryKHR` object when building the BLAS in `main.cpp`:

~~~ cpp
    geometry.flags                              = VK_GEOMETRY_OPAQUE_BIT_KHR | VK_GEOMETRY_NO_DUPLICATE_ANY_HIT_INVOCATION_BIT_KHR;
~~~

`out.hdr` would then look like this:

![](images/8-noDuplicateAnyHit.png)

In a full rendering engine, the body of the `rayQueryProceedEXT` `while` loop would be used to implement
things like **any hit shaders**: pieces of code that the GPU runs for any intersection along the ray.
For instance, imagine someone wants to make parts of a triangle opaque, and others invisible. They could
use `rayQueryConfirmIntersectionEXT` inside this `while` loop to accept any intersection where the
triangle is opaque. This body could also be used with AABB instances to report intersections with non-triangle-mesh
geometry using `rayQueryGenerateIntersectionEXT`. For an example, please see the Ray Query Example in https://www.khronos.org/blog/ray-tracing-in-vulkan.

!!! Tip
    See [https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR/blob/master/ray_tracing_anyhit/README.md](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR/blob/master/ray_tracing_anyhit/README.md)
    for more information about any hit shaders when not using ray queries.

However, tracing non-opaque meshes can be quite slow, as the shader might have to iterate through every
ray-scene intersection. In scenes where there are lots of overlapping objects, this can be relatively expensive.

!!! WARNING
    Revert the changes from this section before continuing, so that the end of the shader looks like this:

    ~~~ glsl
    // Trace the ray and see if and where it intersects the scene!
    // First, initialize a ray query object:
    rayQueryEXT rayQuery;
    rayQueryInitializeEXT(rayQuery,              // Ray query
                          tlas,                  // Top-level acceleration structure
                          gl_RayFlagsOpaqueEXT,  // Ray flags, here saying "treat all geometry as opaque"
                          0xFF,                  // 8-bit instance mask, here saying "trace against all instances"
                          rayOrigin,             // Ray origin
                          0.0,                   // Minimum t-value
                          rayDirection,          // Ray direction
                          10000.0);              // Maximum t-value
  
    // Start traversal, and loop over all ray-scene intersections. When this finishes,
    // rayQuery stores a "committed" intersection, the closest intersection (if any).
    while(rayQueryProceedEXT(rayQuery))
    {
    }
  
    // Get the t-value of the intersection (if there's no intersection, this will
    // be tMax = 10000.0). "true" says "get the committed intersection."
    const float t = rayQueryGetIntersectionTEXT(rayQuery, true);
  
    // Get the index of this invocation in the buffer:
    uint linearIndex = resolution.x * pixel.y + pixel.x;
    // Give the pixel the color (t/10, t/10, t/10):
    imageData[linearIndex] = vec3(t / 10.0);
    ~~~




## Hits and Misses

We can use `rayQueryGetIntersectionTypeEXT` to determine what kind of object the ray intersected, or if the ray
missed the scene entirely. This function takes a ray query (and whether to return the committed or candidate
intersection -- since we won't use the interior of the `rayQueryProceedEXT` `while` loop from here on, this will
always be `true`), and returns a `uint` that tells us if it intersected nothing, a triangle, or generated (procedural) object:

~~~ cpp
uint rayQueryGetIntersectionTypeEXT(rayQueryEXT q, bool committed);
~~~

~~~ cpp
const uint gl_RayQueryCommittedIntersectionNoneEXT = 0U;
const uint gl_RayQueryCommittedIntersectionTriangleEXT = 1U;
const uint gl_RayQueryCommittedIntersectionGeneratedEXT = 2U;
~~~

Let's use this to determine if the ray intersected the scene, or disappeared into the sky. If the ray
hit a triangle, we'll color it green; otherwise, we'll color it dark blue.
Replace the code after the `while` loop with the following:

~~~ glsl
  vec3 pixelColor;
  // Get the type of committed (true) intersection - nothing, a triangle, or
  // a generated object
  if(rayQueryGetIntersectionTypeEXT(rayQuery, true) == gl_RayQueryCommittedIntersectionTriangleEXT)
  {
    // Ray hit a triangle
    pixelColor = vec3(0.0, 1.0, 0.0);
  }
  else
  {
    // Ray hit the sky
    pixelColor = vec3(0.0, 0.0, 0.5);
  }

  // Get the index of this invocation in the buffer:
  uint linearIndex       = resolution.x * pixel.y + pixel.x;
  imageData[linearIndex] = pixelColor;
~~~

Build and run the code. `out.hdr` should now look like this:

![](images/8-hitType.png)

## Triangle Indices

We can also get information about the triangle the ray hit, using `rayQueryGetIntersectionPrimitiveIndexEXT`.

`rayQueryGetIntersectionPrimitiveIndexEXT` takes a ray query (and a committed/candidate boolean), and returns
the **primitive ID** (the index of the triangle), from 0 to `maxPrimitiveCount-1`. (Recall that the first three elements of
`indexBuffer` specified triangle 0, the second three indices specified triangle 1, and so on.)

Here, if the ray hit a triangle, we'll get the index of the triangle. Then we'll give each triangle a
different color. (There are many ways to color different triangles different colors; this
one uses no new techniques.)

Replace the first branch of the `if` statement,

~~~ glsl
    // Ray hit a triangle
    pixelColor = vec3(0.0, 1.0, 0.0);
~~~

with the following code:

~~~ glsl
    const int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);
    pixelColor            = vec3(primitiveID / 10.0, primitiveID / 100.0, primitiveID / 1000.0);
~~~

Build and run the code. `out.hdr` should now look like this:

![](images/8-triangleIndices.png)

## Barycentric Coordinates

Finally, we can determine where the ray intersected the triangle. More specifically,
**barycentric coordinates** tell us where the intersection is, relative to the
vertices of the triangle.

A triangle has three vertices. Let's number them 0, 1, and 2:

*******************************************************
*
*      v0
*       o
*      / \
*     /   \
*    /     \
*   o-------o
* v1         v2
*
*******************************************************

Given an intersection at a point $\mathbf{p}$ in this triangle, `rayQueryGetIntersectionBarycentricsEXT`
returns a `vec2` $\mathbf{b}$ such that 

$$ \mathbf{p} = (1 - b_x - b_y) \textbf{v0} + b_x \textbf{v1} + b_y \textbf{v2}. $$

`1 - b.x - b.y`, `b.x`, and `b.y` are the barycentric coordinates of the intersection, and the equation
above gives a way to get the world-space position of the intersection from the barycentric coordinates.

In other words, `1 - b.x - b.y` is the weight of vertex 0, `b.x` is the weight of vertex 1, and
`b.y` is the weight of vertex 2.

Let's visualize these barycentric coordinates. Replace the first branch of the `if` statement,

~~~ cpp
    const int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);
    pixelColor            = vec3(primitiveID / 10.0, primitiveID / 100.0, primitiveID / 1000.0);
~~~

with the following code:

~~~ cpp
    // Create a vec3(0, b.x, b.y)
    pixelColor = vec3(0.0, rayQueryGetIntersectionBarycentricsEXT(rayQuery, true));
    // Set the first element to 1 - b.x - b.y, setting pixelColor to
    // (1 - b.x - b.y, b.x, b.y).
    pixelColor.x = 1 - pixelColor.y - pixelColor.z;
~~~

Build and run the code. `out.hdr` should now look like this:

![](images/8-barycentricCoordinates.png)

We've now explored four ways to get information from ray queries. However, we haven't covered probably
the most important thing yet: how to get the vertices of the intersected triangle from the ray query.
In the next chapter, we'll make the vertex and index buffers accessible to the compute shader, and use
`rayQueryGetIntersectionPrimitiveIndexEXT` to retrieve vertex data.

<!-- Chapter 9 -->
# Accessing Mesh Data

In this chapter, we'll modify our code to support reading mesh data from triangle intersections. We'll look up triangle vertices, and then use them to compute **triangle normals** (which we'll later use to compute how light bounces off the triangle).

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/10_mesh_data` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Adding Descriptors for the Vertex and Index Buffers

Much like how we added a descriptor for the buffer `buffer`, we'll add storage buffer descriptors for the vertex and index buffers.

!!! Tip Some Different Descriptor Types
    Using a storage buffer technically lets us both read and write the vertex and index buffers. However, we only need to read the vertex and index buffers -- writing to these arrays from the compute shader might be interesting, but we would have to rebuild the acceleration structure in any case.
    
    Instead of a storage buffer, we could also use a **uniform texel buffer** descriptor, which is a read-only descriptor type (that uses an API like reading images in Vulkan). A **storage texel buffer** acts like a storage buffer (it supports both reads and writes), but uses an API like reading and writing images in Vulkan.
    
    Unfortunately, using a **uniform buffer** (like a storage buffer, but read-only) wouldn't make as much sense here, as uniform buffers are limited to about 64 KB of data on desktop devices. (The Cornell Box fits in 64 KB, but this would fail for larger meshes.) We'll show how to send information to the compute shader in a different way, using push constants, in [the extras](extras.html#pushconstants).

First, let's add two new storage buffer bindings to the `nvvk::DescriptorSetContainer`, at the end of the two `descriptorSetContainer.addBinding` calls in `main.cpp`:

~~~ cpp
  descriptorSetContainer.addBinding(2, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(3, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~

To write to these descriptors, we need to update the definition of the `writeDescriptorSets` array.  First
change the declaration of `writeDescriptorSets` to be an array of length 4 instead of an array of length 2,
and then add two new descriptor sets to the end:

~~~ cpp
  // Write values into the descriptor set.
~~~ cpp highlight
  std::array<VkWriteDescriptorSet, 4> writeDescriptorSets;
~~~ cpp
  // 0
  VkDescriptorBufferInfo descriptorBufferInfo{.buffer = buffer.buffer,    // The VkBuffer object
                                              .range  = bufferSizeBytes};  // The length of memory to bind; offset is 0.
  writeDescriptorSets[0] = descriptorSetContainer.makeWrite(0 /*set index*/, 0 /*binding*/, &descriptorBufferInfo);
  // 1
  VkAccelerationStructureKHR tlasCopy = raytracingBuilder.getAccelerationStructure();  // So that we can take its address
  VkWriteDescriptorSetAccelerationStructureKHR descriptorAS{.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET_ACCELERATION_STRUCTURE_KHR,
                                                            .accelerationStructureCount = 1,
                                                            .pAccelerationStructures    = &tlasCopy};
  writeDescriptorSets[1] = descriptorSetContainer.makeWrite(0, 1, &descriptorAS);
~~~ cpp highlight
  // 2
  VkDescriptorBufferInfo vertexDescriptorBufferInfo{.buffer = vertexBuffer.buffer, .range = VK_WHOLE_SIZE};
  writeDescriptorSets[2] = descriptorSetContainer.makeWrite(0, 2, &vertexDescriptorBufferInfo);
  // 3
  VkDescriptorBufferInfo indexDescriptorBufferInfo{.buffer = indexBuffer.buffer, .range = VK_WHOLE_SIZE};
  writeDescriptorSets[3] = descriptorSetContainer.makeWrite(0, 3, &indexDescriptorBufferInfo);
~~~ cpp
  vkUpdateDescriptorSets(context,                                            // The context
                        static_cast<uint32_t>(writeDescriptorSets.size()),  // Number of VkWriteDescriptorSet objects
                        writeDescriptorSets.data(),                         // Pointer to VkWriteDescriptorSet objects
                        0, nullptr);  // An array of VkCopyDescriptorSet objects (unused)
~~~

We've now finished all the C++ code for the main section of this tutorial! All of the remaining coding in the main section of this tutorial will take place in `raytrace.comp.glsl`.

## Accessing `vertexBuffer` and `indexBuffer` from the Compute Shader

We can add definitions for these two descriptors in `raytrace.comp.glsl` much like we did for `buffer`.

Add the following code after the definition of `accelerationStructureEXT tlas` to define the layouts of the vertices and indices.
Note that we've used the `scalar` layout qualifier again to tell GLSL that there's no
padding between consecutive elements:

~~~ glsl
layout(binding = 2, set = 0, scalar) buffer Vertices
{
  vec3 vertices[];
};
layout(binding = 3, set = 0, scalar) buffer Indices
{
  uint indices[];
};
~~~

To make sure that we can access these buffers correctly, let's visualize the first vertex of each triangle.
Replace the first branch of the `if` statement in `raytrace.comp.glsl` (`if(rayQueryGetIntersectionTypeEXT(rayQuery, true) == ...)`) with the following:

First, get the ID of the triangle:

~~~ glsl
	// Get the ID of the triangle
    const int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);
~~~

This triangle used indices $3 \cdot \text{primitiveID}$, $3 \cdot \text{primitiveID} + 1$, and $3 \cdot \text{primitiveID} + 2$.
Get the index of the first vertex of the triangle from the `indices` array:

~~~ glsl
	// Get the index of the first vertex of the triangle
    const uint i0 = indices[3 * primitiveID];
~~~

Then get the position of the first vertex from the `vertices` array:

~~~ glsl
	// Get the position of the first vertex
    const vec3 v0 = vertices[i0];
~~~

Finally, turn the vertex position into a color. This line remaps vertex positions in [-2, 2]^3 (3-element vectors with components between -2 and 2 including the endpoints) into
the visible range ([0, 1]^3):

~~~ glsl
	// Return it as a color
    pixelColor = vec3(0.5) + 0.25 * v0;
~~~

You should now be able to build and run the application, and get the following image. This verifies that
we're reading the vertex and index arrays correctly. (Note that each quad of two triangles has the same
first vertex.)

![](images/9-firstVertex.png)

!!! Tip Negative Values in HDR Files
    HDR files can only represent positive values. As it turns out, if some (but not all) values in an
    HDR file are negative, `stbi_write_hdr` will write out interesting patterns of colors other than
    `(0, 0, 0)`, for instance. This is a pitfall to watch out for when using the HDR image format
    for debugging.

## Getting Intersection Positions

We can also compute the location of the intersection in **object space**, using barycentric coordinates.

Here's how to do that:

~~~ glsl
    // Get the ID of the triangle
    const int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);

    // Get the indices of the vertices of the triangle
    const uint i0 = indices[3 * primitiveID + 0];
    const uint i1 = indices[3 * primitiveID + 1];
    const uint i2 = indices[3 * primitiveID + 2];

    // Get the vertices of the triangle
    const vec3 v0 = vertices[i0];
    const vec3 v1 = vertices[i1];
    const vec3 v2 = vertices[i2];

    // Get the barycentric coordinates of the intersection
    vec3 barycentrics = vec3(0.0, rayQueryGetIntersectionBarycentricsEXT(rayQuery, true));
    barycentrics.x    = 1.0 - barycentrics.y - barycentrics.z;

    // Compute the position of the intersection in object space
    const vec3 objectSpaceIntersection = barycentrics.x * v0 + barycentrics.y * v1 + barycentrics.z * v2;

    // Return it as a color
    pixelColor = vec3(0.5) + 0.25 * objectSpaceIntersection;
~~~

You can now build and run the application and get the following result:

![](images/9-worldPositions.png)

!!! Tip World Space and Object Space
    Meshes (objects) specify the locations of their vertices in their own coordinate space, called **object space**.
    However, the camera traces rays in **world space**. We placed meshes in world-space by specifying its translation,
    rotation, scale, and skew, using the **object-to-world matrix**. This was the `instance.transform` matrix we set
    in `main.cpp`.
    
    In the main section of this tutorial, all meshes use the identity transform matrix, so world space and object
    space are the same. However, if we wanted to, we could transform points from object space to world space
    using the 3 x 4 matrix returned from `rayQueryGetIntersectionObjectToWorldEXT`.
    
    Note that transforming points between coordinate systems works differently than transforming directions between
    coordinate systems, which works differently than transforming normals (to be described) between coordinate systems.
    Briefly, points and vectors are transformed by multiplying by a matrix, while normals are transformed by multiplying by the adjugate of the matrix. For more information, see [this post](https://computergraphics.stackexchange.com/a/1506).

!!! Tip
    We could also get the world-space position of the intersection using the t-value of the ray and the equation
    
    $$ \mathbf{r}(t) = \mathbf{o} + t\mathbf{d}. $$
    
    However, when $t$ is large, numerical rounding error can cause an intersection point generated this way to lie
    further off the plane of the triangle.

## Triangle Normals

In the next chapter, we'll describe how light reflects off a triangle. To do this, we'll need to compute a
particular `vec3` that is perpendicular to the plane of the triangle, called the triangle's [**normal**](https://en.wikipedia.org/wiki/Normal_%28geometry%29).

To do this, consider the triangle shown below:

**********************************
*
*            N
*            ^
*     v2     |         v1
*       *----|--------*
*       |    |       /
*       |    |      /
*       |    |     /
*       |    *    /
*       |        /
*       |       /
*       |      /
*       |     /
*       |    /
*       |   /
*       |  /
*       | /
*       |/
*       * v0
*
**********************************

Assuming the triangle isn't degenerate (i.e. the triangle has nonzero surface area),
to find a vector that is perpendicular to $\text{v1} - \text{v0}$ (the direction from `v0` to `v1`)
and $\text{v2} - \text{v0}$ (the direction from `v0` to `v2`), we can take the cross product
of $\text{v1} - \text{v0}$ and $\text{v2} - \text{v0}$:

~~~ glsl
    const vec3 objectNormal = cross(v1 - v0, v2 - v0);
~~~

We usually also normalize the normal (make it have length 1):

~~~ glsl
    const vec3 objectNormal = normalize(cross(v1 - v0, v2 - v0));
~~~

Note that any given triangle (with nonzero surface area) has two unit normals (`objectNormal` and
`-objectNormal`), which point in opposite directions. In the above code, we've chosen the
normal so that the vertices are ordered counterclockwise relative to it:

**********************************
*
*
*            N
*    .------ ^ ------------. 
*   | v2     |         v1   +   
*   |   *----|--------*    /
*   |   |    |       /    / 
*   |   |    |      /    / 
*   |   |    |     /    /
*   |   |    *    /    /
*   |   |        /    /
*   |   |       /    /
*   |   |      /    /
*   |   |     /    /
*   |   |    /    /
*   |   |   /    /
*   |   |  /    /
*   |   | /    /
*   v   |/    /
*       * v0 
*
**********************************

In other words, we've said that front-facing triangles use a counter-clockwise winding order.
This distinction is especially important when the front of a triangle behaves differently
from the back of a triangle.
See Christoph Michel's [Understanding front faces - winding order and normals](https://cmichel.io/understanding-front-faces-winding-order-and-normals)
for more information about winding orders.

Let's put this together to visualize the normal of each triangle.
Replace the code in the first branch of the `if` statement with the following:

~~~ glsl
    // Get the ID of the triangle
    const int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);

    // Get the indices of the vertices of the triangle
    const uint i0 = indices[3 * primitiveID + 0];
    const uint i1 = indices[3 * primitiveID + 1];
    const uint i2 = indices[3 * primitiveID + 2];

    // Get the vertices of the triangle
    const vec3 v0 = vertices[i0];
    const vec3 v1 = vertices[i1];
    const vec3 v2 = vertices[i2];

    // Compute the normal of the triangle in object space, using the right-hand
    // rule. Since our transformation matrix is the identity, object space
    // is the same as world space.
    //    v2       .
    //    |\       .
    //    | \      .
    //    |  \     .
    //    |/  \    .
    //    /    \   .
    //   /|     \  .
    //  L v0----v1 .
    // n
    const vec3 objectNormal = normalize(cross(v1 - v0, v2 - v0));

    // For this chapter, convert the normal into a visible color.
    pixelColor = vec3(0.5) + 0.5 * objectNormal;
~~~

This will generate the following image:

![](images/9-normals.png)

We've now covered all the API we'll need to write a path tracer.
In the next chapter, we'll start out by path tracing a scene with only mirror
surfaces.


<!-- Chapter -->

# Perfectly Specular Reflections

In this chapter, we'll write a path tracer for mirror surfaces.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/11_specular` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Path Tracing Introduction

**Path tracers** trace paths of light through scenes to render images. Most path tracers today use [**geometric optics**](https://en.wikipedia.org/wiki/Geometrical_optics), which assumes that light travels along rays.

Let's look at how light might travel in a more complicated scene than the one in this tutorial. Imagine an outdoors scene lit by the sun, where a camera looks at a glass on a wood table, with a tree in the background. Here's a diagram of such a scene, showing the sun, the atmosphere, the tree, the table, the glass, and the camera.

****************************************************
*
*        ☀
*          
*           
*            
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*                        
*                              \
*                               +/
*                            \  +
*                           --+-+    /
*                            /  +-+-+--
*                          \    |  \ 
*        🎥              --+-+-+-+
*                        /   /  |
*                               |    /   /
*           |  |                +-+-+-+-+--
*           |  |         \   \  |  \   \
*           |  |      --+-+-+-+-+
*           +--+       /   /    |
*      ---+-+--+--+--           |
*         |       |             |
*         |       |             |
*         |       |             |
****************************************************

In this scene, the sun emits light, which can then bounce off objects, scatter through the atmosphere, and refract and shine through translucent and transparent surfaces, before reaching the camera.

For instance, a ray of light emitted by the sun might scatter off particles in the atmosphere, bounce off a branch of the tree, reflect off the glossy surface of the table, refract through the glass, and finally shine through the lens of the camera, after which a sensor records its color and averages it with all the other rays of light that reach the sensor.

****************************************************
*
*        ☀
*         \
*          \
*           \
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫\🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫o---------o🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫|🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*                       |
*                       |      \
*                       |       +/
*                       |    \  +
*                       |   --+-+    /
*                       |    /  +-+-+--
*                       |  \    |  \ 
*        🎥              o-+-+-+-+
*          ^           / /   /  |
*           \         /         |    /   /
*           |\ |     /          +-+-+-+-+--
*           | \|    /    \   \  |  \   \
*           |  o   /  --+-+-+-+-+
*           +--+\ /    /   /    |
*      ---+-+--+-o+--           |
*         |       |             |
*         |       |             |
*         |       |             |
****************************************************

Each light interaction here changes the color of the light in some way, usually by absorbing some part of the light. For instance, scattering through the sky might tint the light blue (by filtering out greener and redder frequencies), and subsequently bouncing off the branch might tint it dark brown.

We've just described a path of light traveling forwards through the scene. Although we could randomly sample rays of light from the sun, most of these rays would miss the camera, so this would be computationally inefficient without further techniques. (For reference, the sun in our solar system emits the equivalent of roughly 10^45 photons every second.)

By applying the proper weighting factors, we can instead trace rays from the camera to the light:

****************************************************
*
*        ☀
*         ^
*          \
*           \
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫\🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫o---------o🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫|🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫🌫
*                       |
*                       |      \
*                       |       +/
*                       |    \  +
*                       |   --+-+    /
*                       |    /  +-+-+--
*                       |  \    |  \ 
*        🎥              o-+-+-+-+
*          \           / /   /  |
*           \         /         |    /   /
*           |\ |     /          +-+-+-+-+--
*           | \|    /    \   \  |  \   \
*           |  o   /  --+-+-+-+-+
*           +--+\ /    /   /    |
*      ---+-+--+-o+--           |
*         |       |             |
*         |       |             |
*         |       |             |
****************************************************

In other words, instead of asking "where do these rays of light go from this light source", we can ask "where does this pixel's light come from?"

So, most path tracing algorithms usually look a bit like this. Each ray in the path tracer carries some information (usually including a color):

* For each pixel (in parallel),
 * Repeat some number of times:
  * Choose a ray through the camera, then loop:
   * Find where the ray intersects the scene;
   * Get information about the intersection;
   * Do the following:
    * Estimate how much direct light reaches the intersection then flows back along the ray, and break out of the loop;
   * Or:
    * Choose a new ray origin and direction, and update the ray's information.
 * Average the pixel's sampled colors in some way.

In the rest of this chapter, we'll show how to implement this method to create a path tracer where:
* we only use one **sample** per pixel (in other words, the second step loops only once);
* all surfaces are perfectly specular (mirror) surfaces. In particular, this means that all light bounces are deterministic -- we'll implement pseudorandom number generation in the next chapter.

First, we'll start by defining our light source. Instead of a sun and atmosphere, our light source will be the entire sky.

!!! Tip Path Tracing both Forwards and Backwards
    This tutorial's path tracer will only trace rays from the camera. However, some path tracers, known as **bidirectional path tracers**, trace paths from both the camera and from light sources, and connect these paths to create full camera-light paths. These are used in production and are an active area of research, but are outside the scope of this tutorial.

!!! Tip Spectral Rendering
    Some path tracers don't treat rays as having RGB colors -- instead, they might trace a ray for a single frequency of light, or for multiple (e.g. more than three) frequencies of light. These are used in production and are an active area of research, but are outside the scope of this tutorial.

!!! Tip Technical Note: Reversing Rays of Light
    Here's the key principle that allows us to reverse the direction of light in
    the light transport situations in this tutorial is that in the mirror and
    Lambertian BRDFs ([bidirectional reflectance distribution function](https://en.wikipedia.org/wiki/Bidirectional_reflectance_distribution_function); informally, the function that defines a material without subsurface scattering that takes two directions, and says how much light from the first direction bounces to the second direction), the input and output directions can be switched and the value will stay the same.
    In other words, the BRDFs are **symmetric**.
    
    There's a bit more to shading than the BRDF; the Kajiya [rendering equation](https://en.wikipedia.org/wiki/Rendering_equation)
    and the reflectance equation both multiply the BRDF by a cosine term
    (the dot product of the incoming light direction and the surface normal),
    which doesn't appear in this chapter, but which we'll encounter in the
    chapter on diffuse materials. Since this cosine term could be confusing
    (since it requires distinguishing between the BRDF and radiance ratios) and
    it's possible to complete the tutorial without defining the BRDF, we mention
    it only as a technical note here.
    
    However, not all BRDFs are symmetric, so forwards light tracing from light
    sources isn't always the same as backwards (importance) tracing from a camera.
    Some cases where this can occur are when the normal used for shading is
    different than the geometry's normal (such as when using normal maps),
    and when the refractive index changes at a material boundary (e.g. imagine
    light refracting through glass). Bidirectional path tracers often have to
    add factors to BRDFs to account for this to avoid artifacts. For a much
    deeper analysis, please see Eric Veach's PhD thesis,
    [*Robust Monte Carlo Methods for Light Transport Simulation*](https://graphics.stanford.edu/papers/veach_thesis/).

## Sky Lighting

Add the following code in `raytrace.comp.glsl`, after defining the `Indices` storage buffer. Recall that in our coordinate space, the positive Y axis points upwards:

~~~ glsl
// Returns the color of the sky in a given direction (in linear color space)
vec3 skyColor(vec3 direction)
{
  // +y in world space is up, so:
  if(direction.y > 0.0f)
  {
    return mix(vec3(1.0f), vec3(0.25f, 0.5f, 1.0f), direction.y);
  }
  else
  {
    return vec3(0.03f);
  }
}
~~~

Here, we use the GLSL `mix` function, which linearly interpolates between its first and second arguments, using the third argument. It is defined by:

$$ \text{mix}(a, b, t) = a (1-t) + b t. $$

In particular, when t = 0, `mix(a, b, t) = a`, and when t = 1, `mix(a, b, t) = b`. In our case, this means that the sky dome:
* is dark gray below the horizon;
* is white near the horizon;
* fades to a blue near the zenith.

(This sky is adapted from [Chapter 4 of Ray Tracing in One Weekend](https://raytracing.github.io/books/RayTracingInOneWeekend.html#rays,asimplecamera,andbackground/sendingraysintothescene).)

## Refactoring Intersection Information

Let's turn the code that takes a ray query and gets information about the intersection point into its own function. This function, `getObjectHitInfo`, will return a `HitInfo` struct, which will contain the **albedo** (diffuse reflective color) of the mesh at the intersection point (which we'll set to (0.7, 0.7, 0.7), a light gray), the position of the intersection in world space, and the normal of the intersection in world space.

Add the following code after the `skyColor` definition:

~~~ glsl
struct HitInfo
{
  vec3 color;
  vec3 worldPosition;
  vec3 worldNormal;
};

HitInfo getObjectHitInfo(rayQueryEXT rayQuery)
{
  HitInfo result;
  // Get the ID of the triangle
  const int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);

  // Get the indices of the vertices of the triangle
  const uint i0 = indices[3 * primitiveID + 0];
  const uint i1 = indices[3 * primitiveID + 1];
  const uint i2 = indices[3 * primitiveID + 2];

  // Get the vertices of the triangle
  const vec3 v0 = vertices[i0];
  const vec3 v1 = vertices[i1];
  const vec3 v2 = vertices[i2];

  // Get the barycentric coordinates of the intersection
  vec3 barycentrics = vec3(0.0, rayQueryGetIntersectionBarycentricsEXT(rayQuery, true));
  barycentrics.x    = 1.0 - barycentrics.y - barycentrics.z;

  // Compute the coordinates of the intersection
  const vec3 objectPos = v0 * barycentrics.x + v1 * barycentrics.y + v2 * barycentrics.z;
  // For the main tutorial, object space is the same as world space:
  result.worldPosition = objectPos;

  // Compute the normal of the triangle in object space, using the right-hand rule:
  //    v2      .
  //    |\      .
  //    | \     .
  //    |/ \    .
  //    /   \   .
  //   /|    \  .
  //  L v0---v1 .
  // n
  const vec3 objectNormal = normalize(cross(v1 - v0, v2 - v0));
  // For the main tutorial, object space is the same as world space:
  result.worldNormal = objectNormal;

  result.color = vec3(0.7f);

  return result;
}
~~~

Remove the code in `main()` from the `// Trace the ray` comment up to, but not including, the `uint linearIndex` line.

## Tracing a Path

Now, let's implement the main loop of the ray tracer.

After defining `rayDirection` in `main()`, add the following line to normalize the ray direction (make it have a length of 1). This makes future math simpler than it would otherwise be:

~~~ glsl
  rayDirection                 = normalize(rayDirection);
~~~

Then add a `vec3 accumulatedRayColor`, which will track the "color" of the ray so far (in other words, what percentage of red, green, and blue light hasn't been absorbed yet). Also add the pixel color, which will default to (0, 0, 0) in case we trace a ray for long enough without reaching the sky.

~~~ glsl
  vec3 accumulatedRayColor = vec3(1.0);  // The amount of light that made it to the end of the current ray.
  vec3 pixelColor          = vec3(0.0);
~~~

Now begin the inner loop. We'll iterate over at most 32 path segments (31 bounces), before we decide that it isn't worth the additional effort to trace the ray further. (Most rays will reach the sky before then.)

~~~ glsl
  // Limit the kernel to trace at most 32 segments.
  for(int tracedSegments = 0; tracedSegments < 32; tracedSegments++)
  {
~~~

Trace the ray:

~~~ glsl
    // Trace the ray and see if and where it intersects the scene!
    // First, initialize a ray query object:
    rayQueryEXT rayQuery;
    rayQueryInitializeEXT(rayQuery,              // Ray query
                          tlas,                  // Top-level acceleration structure
                          gl_RayFlagsOpaqueEXT,  // Ray flags, here saying "treat all geometry as opaque"
                          0xFF,                  // 8-bit instance mask, here saying "trace against all instances"
                          rayOrigin,             // Ray origin
                          0.0,                   // Minimum t-value
                          rayDirection,          // Ray direction
                          10000.0);              // Maximum t-value

    // Start traversal, and loop over all ray-scene intersections. When this finishes,
    // rayQuery stores a "committed" intersection, the closest intersection (if any).
    while(rayQueryProceedEXT(rayQuery))
    {
    }
~~~

Then see if the ray hit a triangle, or if it hit the sky:

~~~ glsl
    // Get the type of committed (true) intersection - nothing, a triangle, or
    // a generated object
    if(rayQueryGetIntersectionTypeEXT(rayQuery, true) == gl_RayQueryCommittedIntersectionTriangleEXT)
    {
~~~

If it hit a triangle, then we get the color (albedo), world position, and world normal of the intersection point.

~~~ glsl
      // Ray hit a triangle
      HitInfo hitInfo = getObjectHitInfo(rayQuery);
~~~

We apply color absorption by multiplying the albedo with the current ray color. (For example, a perfectly blue surface might only reflect blue light; this is equivalent to multiplying the accumulated ray color by something like `vec3(0, 0, 0.9)`.)
~~~ glsl
      // Apply color absorption
      accumulatedRayColor *= hitInfo.color;
~~~

Now, we must reflect the ray off the surface. Here's a diagram to show how this happens -- we start a new ray at the intersection point, and reflect the direction across the normal $N$:

***********************************
*
* Old ray   N   New ray
*     \     ^     ^
*      \    |    /
*       \   |   / |
*        \  |  /
*         \ | /   |
*          v|/
*-----------o-----|-----
*            \    
*                 | parallel to N
*              \
*                 |
*                \
*                 v
*
***********************************

GLSL has a built-in function for calculating the new ray direction, called [`reflect`](https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/reflect.xhtml). Add this code to use it:

~~~ glsl
      // Start a new ray at the hit position:
      rayOrigin = hitInfo.worldPosition;

      // Reflect the direction of the ray using the triangle normal:
      rayDirection = reflect(rayDirection, hitInfo.worldNormal);
    }
~~~

!!! Warning
    The `rayOrigin` line here has a small bug. We'll reveal what it is and fix it in the next section.

Otherwise, the ray hit the sky, so get the sky color, multiply it by `accumulatedRayColor` (the percentage of light that made it to the end of the ray without getting absorbed) to set `pixelColor`, and break out of the loop:

~~~ glsl
    else
    {
      // Ray hit the sky
      pixelColor = accumulatedRayColor * skyColor(rayDirection);
      break;
    }
  }
~~~

That's all that's needed here! You should now be able to compile and run it, and see this:

![](images/10-reflectionBug.png)

We have reflections, but most of the surfaces don't look realistic. Let's fix that.

## Offsetting the Ray Origin

!!! Tip
    To skip this section, change the `rayOrigin` line to
    ~~~ glsl
      // Flip the normal so it points against the ray direction:
      hitInfo.worldNormal = faceforward(hitInfo.worldNormal, rayDirection, hitInfo.worldNormal);

      // Start a new ray at the hit position, but offset it slightly along the normal:
      rayOrigin = hitInfo.worldPosition + 0.0001 * hitInfo.worldNormal;
    ~~~

The bug here is that we set the minimum t-value of the `rayQueryInitializeEXT` call to 0, but when the ray starts on (or slightly on the wrong side of) a surface, it might intersect the surface again just where the ray started, getting stuck in the surface.

*************************
*
* Old ray   N
*     \     ^
*      \    |
*       \   |
*        \  |
*         \ |
*          v| Gets stuck
*-----------o-----------
*          ^
*         /
*        /
*
*************************

There are many different solutions to this problem. Wächter and Binder describe probably the state-of-the-art approach in their
article "A Fast and Robust Method for Avoiding Self-Intersection" in the free book [Ray Tracing Gems](http://www.realtimerendering.com/raytracinggems/), which uses about 18 lines of code.

For this tutorial, it will suffice to move the ray origin a small distance away from the intersection, along the surface normal:

~~~ glsl
      // Start a new ray at the hit position, but offset it slightly along
      // the normal:
      rayOrigin = hitInfo.worldPosition + 0.0001 * hitInfo.worldNormal;
~~~

***********************************
*
* Old ray   N  New ray
*     \     ^   ^
*      \    |  /
*       \   | /
*        \  |/
*         \ o 
*          v|
*-----------o-----------
*
***********************************

You should now be able to compile and run it again, and see this:

![](images/10-reflectionPt2.png)

However, there's one more thing we can do. Recall that non-degenerate triangles have two normals. But what happens if a ray hits a triangle from the wrong side? For instance, this can happen with the rectangle in the center of the top of the box, if a ray bounces off the ceiling and behind the rectangle. With our current method, the new ray will be pushed through the surface, then reflect against the surface again. (However, this is nearly imperceptible in our image.) Here's a zoomed-in view, showing two bounces:

*************************
*
*           N
*           ^     ^
*           |    /
*           |   /
*           o  o  
*           |\   
*           | v 
*-----------o--o--------
*          ^
*         /
*        /
*       /
*      /
*     /
* Old ray
*
*************************

In some sense, this simulates a one-sided surface, where one side is opaque, but the other is translucent. (One-sided surfaces are often used for artistic effects, but there are more efficient ways to implement them in ray tracing, such as by using backface culling.)

To solve this, we can simulate a two-sided surface: we offset the ray origin along the normal, against the direction of the old ray.

To do this, we can use GLSL's [`faceforward`](https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/faceforward.xhtml) function:

~~~ glsl
genType faceforward( genType N,
                     genType I,
                     genType Nref);
~~~

(in our case, `genType` is `vec3`). This function returns `N` or `-N` depending on the directions `I` (like an incoming light vector) and `Nref` (like a normal of a surface). If `I` and `Nref` point towards each other (i.e. their angles differ by more than 90 degrees), then this returns `N`. Otherwise (if `I` and `Nref` point within 90 degrees of the same direction), then this returns `-N`.

So, replace the `rayOrigin` line with

~~~ glsl
        // Flip the normal so it points against the ray direction:
        hitInfo.worldNormal = faceforward(hitInfo.worldNormal, rayDirection, hitInfo.worldNormal);

        // Start a new ray at the hit position, but offset it slightly along the normal:
        rayOrigin = hitInfo.worldPosition + 0.0001 * hitInfo.worldNormal;
~~~


<!-- Custom admonition to handle diagrams -->
<div class ="admonition tip">
<div class="admonition-title">The Dot Product</div>
Internally, `faceforward` uses the [**dot product**](https://en.wikipedia.org/wiki/Dot_product) of two vectors. This can be calculated using

$$ \text{dot}((x_1, y_1, z_1), (x_2, y_2, z_2)) = x_1 x_2 + y_1 y_2 + z_1 z_2 $$

and gives the product of the length of the two vectors, and the cosine of the angle between the two vectors. In particular, two vectors point within 90 degrees of each other if their dot product is positive, and are separated by more than 90 degrees (i.e. "face different directions") if their dot product is negative.

***************************************
*
*           N       
*           ^      
*           |     
*           |    
*           |            dot(N, I) < 0
*           |  
*           +.           dot(N, I)
*-----------o | θ ------ =
*           ++           |N||I| cos(θ)
*           |
*           |
*           |
*           |
*           v
*           I
*
***************************************

***************************************
*
*           N      I
*           ^     ^
*           | θ  /       
*           |---/        dot(N, I) > 0
*           |  /         
*           | /          dot(N, I)
*           |/           =
*-----------o----------- |N||I| cos(θ)
*
***************************************

In particular, if $N$ and $I$ are unit vectors (have length 1), then $\text{dot}(N, I) = \cos(θ)$, the cosine of the angle between them, because $|N| = |I| = 1$.

So, `faceforward(N, I, Nref)` returns `N` if `dot(I, Nref) < 0`, and `-N` otherwise.
</div>

You should now have a path tracer that renders reflective surfaces, with a sky in the background! Like a hall of mirrors, you can see many levels of reflections inside reflections -- something that would be difficult to do as concisely and quickly as with a ray tracer.

![](images/10-reflectionPt3.png)

Here's the difference between the two images:

![Note that this image is relatively dark; look near the top-center to see the differences.](images/10-reflectionDiff.png)

One may notice that the edges of objects are jagged rather than smooth. (Including the left wall -- the left edge of the mesh is not perfectly vertical.) In the next chapter, we'll explore how to use pseudorandom number generation to smooth out these edges and make the image look as if it was rendered at higher resolution by **antialiasing** the image.


<!-- Chapter -->

# Antialiasing and Pseudorandom Number Generation

The jagged edges we saw at the end of the last chapter are one kind of [**aliasing**](https://en.wikipedia.org/wiki/Aliasing) artifact. Antialiasing is the process of removing aliasing artifacts.

There are many ways of thinking about antialiasing, and researchers frequently come up with new ways of approaching the problem. Here's one way: In some sense, we're trying to represent an infinite-resolution image (what the virtual camera sees) with a finite-resolution image (the 2D array of colors). However, the way we're currently downsampling this infinite-resolution image into a finite-resolution image -- by only measuring incoming light once, at the center of each pixel -- isn't very good.

To solve this, we'll evaluate the incoming light at multiple samples per pixel, then average them to get a smoother image. We'll choose the sample locations -- the places where each pixel evaluates incoming light -- by creating a pseudorandom number generator (PRNG), like a GLSL version of `std::rand()`.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/12_antialiasing` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Pseudorandom Number Generation in GLSL

We'll use a version of `pcg32i_random_t` from the [PCG random number generator library](https://www.pcg-random.org/index.html) to generate numbers that look random. This will take about 16 lines of code.

`pcg32i_random_t` uses a 32-bit unsigned integer state. We seed the PRNG by setting this state to an initial value. Each pixel will have a different seed. We can step this state forwards to get a new random state, and convert the PRNG state to a floating-point number between 0 and 1 using a version of `pcg_output_rxs_m_xs_32_32`.

We'll write two functions: `stepRNG` updates the RNG state, and `stepAndOutputRNGFloat` updates the RNG state in-place while returning a `float` between 0 and 1.

****************************************
*
*       .-------.
*       | Seed  |
*       '-------'
*           |
* stepAndOutputRNGFloat
*           |      |
*           v       '----> random float
*       .-------.
*       | State |
*       '-------'
*           |
* stepAndOutputRNGFloat
*           |      |
*           v       '----> random float
*       .-------.
*       | State |
*       '-------'
*           |
* stepAndOutputRNGFloat
*           |      |
*           v       '----> random float
*
*          ...
*
****************************************

In `raytrace.comp.glsl`, add these two functions after defining the `Indices` storage buffer:

~~~ glsl
// Random number generation using pcg32i_random_t, using inc = 1. Our random state is a uint.
uint stepRNG(uint rngState)
{
  return rngState * 747796405 + 1;
}

// Steps the RNG and returns a floating-point value between 0 and 1 inclusive.
float stepAndOutputRNGFloat(inout uint rngState)
{
  // Condensed version of pcg_output_rxs_m_xs_32_32, with simple conversion to floating-point [0,1].
  rngState  = stepRNG(rngState);
  uint word = ((rngState >> ((rngState >> 28) + 4)) ^ rngState) * 277803737;
  word      = (word >> 22) ^ word;
  return float(word) / 4294967295.0f;
}
~~~

`stepAndOutputRNGFloat` uses the `inout` qualifier, which acts like a reference in C++ -- that is, it can modify `rngState` in-place.

For now, let's visualize the random noise this generates. After the `if((pixel.x >= ...))` block, add the state for the random number generator:

~~~ glsl
  // State of the random number generator.
  uint rngState = resolution.x * pixel.y + pixel.x;  // Initial seed
~~~

Then change the last few lines of `main()` to step, then directly output the PRNG state:

~~~ glsl
  pixelColor = vec3(stepAndOutputRNGFloat(rngState));

  // Get the index of this invocation in the buffer:
  uint linearIndex       = resolution.x * pixel.y + pixel.x;
  imageData[linearIndex] = pixelColor;
~~~

You should now be able to build and run the program, and get an image that looks like this:

![](images/11-randomNoise.png)

!!! WARNING
    Remove the `pixelColor = vec3(stepAndOutputRNGFloat(rngState));` line before continuing.

## Antialiasing

To perform antialiasing, we'll take multiple samples around the center of each pixel. But how should we choose these sample locations?

The answer is to choose sample locations that mimic a low-pass filter, such as a box or Gaussian kernel (i.e. a very small blurring kernel, or a filter that attempts to filter out very high frequencies with wavelengths smaller than two pixel distances).

In this chapter, we'll implement a box filter by choosing random locations within half a pixel of each pixel center. (This averages out all incoming light within a box around each pixel, which is why it's called a box filter). In [the extras](extras.html#gaussianfilterantialiasing), we'll see how to use some additional mathematics to create a higher-quality Gaussian filter.

!!! Tip
    Digital cameras also must perform some sort of antialiasing to avoid aliasing artifacts -- but instead of taking and averaging random samples (although some cameras perform a version of this by applying small vibrations to the sensor), most use an optical low-pass filter which applies the equivalent of a small-scale blur to the light itself before it reaches the sensor. (See [this section](https://en.wikipedia.org/wiki/Anti-aliasing_filter#Optical_applications) for more information.)

!!! Tip What is the Perfect Antialiasing Filter?
    Perhaps surprisingly, the answer to this question is much more complex than it initially appears.
    
    From a perceptual point of view, a goal of antialiasing is to make a
    finite-resolution version of an infinite-resolution image feel as close as
    possible to the infinite-resolution image, when viewed by a human on a monitor.
    This introduces an enormous amount of additional complexity, and the answer
    to this perceptual problem may never be known.
    
    From a signal processing point of view, spatial aliasing happens when we try
    to sample an image with frequencies that are too high to be reconstructed
    from the sampled image. The ideal low-pass filter is the
    [sinc filter](https://en.wikipedia.org/wiki/Sinc_filter). This filter has
    negative weights at some points -- we don't use sample weights in this
    tutorial, but we could implement this filter by giving each sample a
    (possibly negative) weight, then dividing by the sum of the weights.
    However, the sinc filter is rarely used in computer graphics because these
    negative values produce "ringing" effects around edges. Instead, most
    renderers use something like a Gaussian or spline-based filter.
    
    In particular, looking at antialiasing as a signal reconstruction problem
    means that a square-shaped box filter is usually not the ideal antialiasing
    filter. Nevertheless, we'll show how to implement a box filter in this main
    tutorial, and then show how to change this to a higher-quality Gaussian
    filter in [the extras](extras.html#raytracingpipelines/overview/shaderbindingtables).
    For a longer explanation, please see Alvy Ray Smith's technical memo
    [*A Pixel Is Not A Little Square...*](http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf),
    which also discusses some cases when it makes sense to use a box filter.
    Close to this topic is **subpixel antialiasing**, which can be especially
    complex on screens where subpixels aren't rounded rectangles such as [PenTile](https://en.wikipedia.org/wiki/PenTile_matrix_family).
    
    When the image is noisy (as will be the case in the next chapter), using
    variants of **blue noise** and low-discrepancy sequences for sampling helps
    make the image converge faster, in addition to several other nice properties.
    This continues to be an area of active research; for instance, see A. Cengiz
    Öztireli's paper [A Comprehensive Theory and Variational Framework for Anti-aliasing Sample Patterns](https://arxiv.org/abs/1902.08228) from EGSR 2020.

Let's use our PRNG to implement antialiasing. We'll change the middle of `main()` to perform the following steps:

- Initialize the PRNG;
- Set the camera parameters;
- Set the sum of sample colors to 0;
- Repeat for 64 samples:
 - Choose a random pixel center;
 - Compute the ray through the pixel center;
 - Ray trace the ray and get the color along the ray;
 - Add the color to the sum of sample colors.
- Output the average pixel color, which is the sum of sample colors divided by the number of samples.

To do this, change the body of `main()` after the `if((pixel.x >= ...))` block to the following.

~~~ glsl
  // State of the random number generator.
  uint rngState = resolution.x * pixel.y + pixel.x;  // Initial seed

  // This scene uses a right-handed coordinate system like the OBJ file format, where the
  // +x axis points right, the +y axis points up, and the -z axis points into the screen.
  // The camera is located at (-0.001, 1, 6).
  const vec3 cameraOrigin = vec3(-0.001, 1.0, 6.0);
  // Define the field of view by the vertical slope of the topmost rays:
  const float fovVerticalSlope = 1.0 / 5.0;

  // The sum of the colors of all of the samples.
  vec3 summedPixelColor = vec3(0.0);

  // Limit the kernel to trace at most 64 samples.
  const int NUM_SAMPLES = 64;
  for(int sampleIdx = 0; sampleIdx < NUM_SAMPLES; sampleIdx++)
  {
    // Rays always originate at the camera for now. In the future, they'll
    // bounce around the scene.
    vec3 rayOrigin = cameraOrigin;
    // Compute the direction of the ray for this pixel. To do this, we first
    // transform the screen coordinates to look like this, where a is the
    // aspect ratio (width/height) of the screen:
    //           1
    //    .------+------.
    //    |      |      |
    // -a + ---- 0 ---- + a
    //    |      |      |
    //    '------+------'
    //          -1
    const vec2 randomPixelCenter = vec2(pixel) + vec2(stepAndOutputRNGFloat(rngState), stepAndOutputRNGFloat(rngState));
    const vec2 screenUV          = vec2((2.0 * randomPixelCenter.x - resolution.x) / resolution.y,    //
                               -(2.0 * randomPixelCenter.y - resolution.y) / resolution.y);  // Flip the y axis
    // Create a ray direction:
    vec3 rayDirection = vec3(fovVerticalSlope * screenUV.x, fovVerticalSlope * screenUV.y, -1.0);
    rayDirection      = normalize(rayDirection);

    vec3 accumulatedRayColor = vec3(1.0);  // The amount of light that made it to the end of the current ray.

    // Limit the kernel to trace at most 32 segments.
    for(int tracedSegments = 0; tracedSegments < 32; tracedSegments++)
    {
      // Trace the ray and see if and where it intersects the scene!
      // First, initialize a ray query object:
      rayQueryEXT rayQuery;
      rayQueryInitializeEXT(rayQuery,              // Ray query
                            tlas,                  // Top-level acceleration structure
                            gl_RayFlagsOpaqueEXT,  // Ray flags, here saying "treat all geometry as opaque"
                            0xFF,                  // 8-bit instance mask, here saying "trace against all instances"
                            rayOrigin,             // Ray origin
                            0.0,                   // Minimum t-value
                            rayDirection,          // Ray direction
                            10000.0);              // Maximum t-value

      // Start traversal, and loop over all ray-scene intersections. When this finishes,
      // rayQuery stores a "committed" intersection, the closest intersection (if any).
      while(rayQueryProceedEXT(rayQuery))
      {
      }

      // Get the type of committed (true) intersection - nothing, a triangle, or
      // a generated object
      if(rayQueryGetIntersectionTypeEXT(rayQuery, true) == gl_RayQueryCommittedIntersectionTriangleEXT)
      {
        // Ray hit a triangle
        HitInfo hitInfo = getObjectHitInfo(rayQuery);

        // Apply color absorption
        accumulatedRayColor *= hitInfo.color;

        // Start a new ray at the hit position, but offset it slightly along
        // the normal against rayDirection:
        rayOrigin = hitInfo.worldPosition - 0.0001 * sign(dot(rayDirection, hitInfo.worldNormal)) * hitInfo.worldNormal;

        // Reflect the direction of the ray using the triangle normal:
        rayDirection = reflect(rayDirection, hitInfo.worldNormal);
      }
      else
      {
        // Ray hit the sky
        accumulatedRayColor *= skyColor(rayDirection);
        
        // Sum this with the pixel's other samples.
        // (Note that we treat a ray that didn't find a light source as if it had
        // an accumulated color of (0, 0, 0)).
        summedPixelColor += accumulatedRayColor;
    
        break;
      }
    }
  }

  // Get the index of this invocation in the buffer:
  uint linearIndex       = resolution.x * pixel.y + pixel.x;
  imageData[linearIndex] = summedPixelColor / float(NUM_SAMPLES);  // Take the average
~~~
    
Note that we've modified and moved the logic for generating new camera rays into the outermost `for` loop.

You should now be able to build and run the project, and see that the edges in the reflections are now much smoother, at the expense of some additional noise!

![](images/11-antialiased.png)

!!! Tip Why only 64 samples per pixel?
    This image has some noise on the edges instead of aliasing artifacts, and
    increasing the number of samples per pixel would reduce the amount of noise.
    However, rendering significantly more samples per pixel (on the order of
    thousands on current hardware) could make the command buffer take longer
    than two seconds to execute. On Windows, this could cause 
    Windows' [Timeout Detection and Recovery (TDR)](https://docs.microsoft.com/en-us/windows-hardware/drivers/display/timeout-detection-and-recovery)
    system to think that the program was stuck in an infinite loop, and reset the GPU driver.
    
    In [the extras](extras.html#moresamples), we'll show how to use multiple
    command buffers to perform as many samples as wanted, without running into TDR.
    
    Additionally, most real-time ray tracing applications today might use one,
    or even an average of less than one, sample per pixel. They usually do this
    by using better sampling techniques, denoisers that remove noise without
    performing additional samples, and upscalers that scale a low-resolution
    rendered image to high-resolution, while approximating or improving on
    native high-resolution image quality.


<!-- Chapter -->

# Diffuse Reflection

In reality, not all surfaces are mirror surfaces. On a perfectly diffuse surface, light scatters apparently randomly in all directions away from the surface. On a glossy surface, light scatters more closely, but still randomly, to a perfect reflection, resulting in a blurry reflection. Most materials are complex combinations of these different types of reflection, including light interactions we haven't talked about yet.

In this chapter, we'll complete the path tracer by giving the mesh a perfectly diffuse material. We'll be able to see shadows, diffuse reflections, and much more. And, perhaps surprisingly, this will only use 10 new lines of code, five of which are comments.

!!! Tip
    You can find the reference code at the end of this chapter in the `vk_mini_path_tracer` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## How Should Light Reflect?

In this chapter, we'll implement a **Lambertian** (or **perfectly diffuse**) **material model**.
A surface with a Lambertian material appears to have the same shading no matter
where one views it from (so long as they can see the surface).

When we render a perfect sphere with a Lambertian material under a perfectly directional
light source, we see the shading fall off from the point where the normal faces
most into the directional light, to the ring where the normals are perpendicular
to the light direction. The surface below that ring is shadowed.

![A sphere with a Lambertian material under unidirectional lighting.](images/13-lambertSphere.png)

Here's why this happens. Imagine a set of parallel rays of light traveling through space.

*********************************
*
* ---------------------------->
* ---------------------------->
* ---------------------------->
* ---------------------------->
* ---------------------------->
*
*********************************

When these rays hit a surface angled towards the light (i.e. with a normal pointing towards the light), each unit of surface area receives some amount of light.

*********************************
*
*               |
*               |
* ------------->|
* ------------->|
* ------------->|
* ------------->|
* ------------->|
*               |
*
*********************************

When this surface is angled away from the light, each unit of surface area receives less light.

*********************************
*
*                     /
* ------------------>/
* ----------------->/
* ---------------->/
* --------------->/
* -------------->/
*               /
*
*********************************

Finally, when the normal is angled perpendicularly to the light, each unit of surface area receives no light.

*********************************
*
* ---------------------------->
* ---------------------------->
*        ---------------
* ---------------------------->
* ---------------------------->
*
*********************************

As a result, when tracing light rays from the camera and hitting a Lambertian
surface, light is more likely to come from a direction closer to the surface normal.

!!! Tip Technical Note
    More specifically, if one imagined the sphere above as being covered by
    infinitesimally small patches with equal surface area, then the light
    received by each patch would be proportional to the cosine of the angle
    between the surface normal and the direction to the directional light.
    
    This is the **cosine term**, $\omega_i \cdot \mathbf{n} = \cos(\theta_i)$ in the [Kajiya rendering equation](https://en.wikipedia.org/wiki/Rendering_equation).

This path tracer traces rays of light backwards, from the camera to a light
source. To choose the next ray in a path, we need to randomly choose an incoming
ray direction. However, some directions are more likely than others. For a
Lambertian surface, we'll need to choose rays closer to the normal more often on
average than rays closer to the horizon, as depicted below.

![Diagram of the distribution of incoming light (yellow) for a Lambertian surface.
In this context, the blue ray is the vector from the camera to the intersection
point, and the length of each yellow ray is proportional to the probability that
the material model chooses that ray as the incoming light direction.](images/13-lambertBkwds.png)

For an excellent, more in-depth exploration of how light interacts with surfaces, please see Bartosz Ciechanowski's article [*Lights and Shadows*](https://ciechanow.ski/lights-and-shadows/).

!!! Tip
    For a Lambertian surface, the polar plot above also shows how light scatters
    off the surface -- not uniformly in all directions, but with a cosine term.
    However, tracing light rays forward from a light isn't always the same as
    tracing importance rays backwards from a camera with more complex material
    models -- for more information, see *Technical Note: Reversing Rays of Light*
    in Section [Path Tracing Introduction].

## Choosing a Lambertian Reflected Light Direction

It turns out there's a way to perfectly sample this distribution, so that we choose directions closer to the normal more often than other directions. (This comes from [chapter 8.5](https://raytracing.github.io/books/RayTracingInOneWeekend.html#diffusematerials/truelambertianreflection) of Ray Tracing in One Weekend.)

Imagine the surface, the normal at the intersection point, and a sphere of radius 1 centered on the normal.

![A surface, a normal at an intersection point, and a sphere (here represented by a circle) centered at that normal of radius 1.](images/13-randomSample.svg)

To sample a random Lambertian reflection direction, choose a random point on the sphere, then normalize it; this gives the needed distribution!

However, sampling a random point on the sphere is a bit more complex. To do this, we'll generate two random numbers $a$ and $b$ between 0 and 1. Then we'll compute

$$ \begin{aligned} \theta &= 2 \pi a \\ u &= 2 b - 1 \\ r &= \sqrt{1 - u^2} \\ \textbf{p} &= \left(r \cos(\theta), r \sin(\theta), u\right). \end{aligned} $$

$\textbf{p}$ is then a random point on the unit sphere centered at (0,0,0). We then add the world-space normal, then normalize, to get the reflected ray direction.

To implement this, replace the reflection code in `raytrace.comp.glsl`:

~~~ glsl
        // Reflect the direction of the ray using the triangle normal:
        rayDirection = reflect(rayDirection, hitInfo.worldNormal);
~~~

with:

~~~ glsl
        // For a random diffuse bounce direction, we follow the approach of
        // Ray Tracing in One Weekend, and generate a random point on a sphere
        // of radius 1 centered at the normal. This uses the random_unit_vector
        // function from chapter 8.5:
        const float theta = 6.2831853 * stepAndOutputRNGFloat(rngState);   // Random in [0, 2pi]
        const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;  // Random in [-1, 1]
        const float r     = sqrt(1.0 - u * u);
        rayDirection      = hitInfo.worldNormal + vec3(r * cos(theta), r * sin(theta), u);
        // Then normalize the ray direction:
        rayDirection = normalize(rayDirection);
~~~

That's all you need for a Lambertian material model. You should now be able to build and run the code, and see the following:

![](images/12-vk_mini_path_tracer.png)

With 10 lines of code, we've gone from an entirely reflective scene to a diffuse render. This render includes effects such as soft shadows, diffuse reflection, ambient occlusion, and area lighting (the sky). It should also match the output of other ray tracing programs such as Blender if the sky and Lambertian surface material are set up correctly. Adding more samples per pixel will reduce the level of noise. At the same time, because of the GPU acceleration techniques we've used, this image is fast to render.

This completes the tutorial!

## Summary

Congratulations! You've now built a GPU-accelerated path tracer in Vulkan. There were many steps along the way, from initializing Vulkan to the final diffuse material model, but the final C++ code should be less than 300 lines long.

To summarize, this program:

- Initializes an `nvvk::Context` with the `VK_KHR_ray_query` and `VK_KHR_acceleration_structure` extension (Chapter [Device Extensions and Vulkan Objects]);
- Allocates a buffer for image data (Section [Vulkan Buffers]);
- Loads an OBJ file (Section [Loading an OBJ File]);
- Creates an command pool (Section [Command Buffers and Pools]);
- Uploads vertex and index buffers to the GPU (Section [Uploading Data to the GPU]);
- Turns the mesh data into a BLAS and TLAS (Section [Acceleration Structures]);
- Creates a descriptor set that references the color buffer, acceleration structure, and vertex and index buffers (Chapter [Accessing Mesh Data]);
- Loads and compiles the compute shader, and creates the compute pipeline that uses ray queries (Chapter [Compute Shaders]);
- Starts recording a command buffer:
 - Binds the compute shader pipeline and descriptor set (Section [Binding the Descriptor Set and Dispatching])
 - Runs the compute shader pipeline (Section [Binding the Descriptor Set and Dispatching]);
 - Adds a pipeline barrier to make sure the CPU can access the written data (Section [Synchronization, Part 1: A Pipeline Barrier]);
- Ends and submits the command buffer and waits for it to finish (Section [Refactoring Command Buffers]);
- Writes the color buffer to an HDR file (Chapter [Writing an Image]);
- and finally, destroys all objects.

It's sometimes useful for me to visualize the dependencies between the Vulkan objects. Here's a graph of all the Vulkan and NVVK objects we created (aside from `nvvk::Context`, since nearly every object relies on it), as well as the final command buffer commands:

*************************************************************************************
*
*  .---------------.              .--------------------------------------.
*  | VkCommandPool |              |   nvvk::ResourceAllocatorDedicated   |
*  '-------+-------'              '-+------------------+---------------+-'
*          |                        |                  |               |
*          |                        v                  v               |
*          |           .--------------.  .--------------------------.  |
*          |           | Color buffer |  | Vertex and index buffers |  |
*          |           '------------+-'  '------+-----------+-------'  |
*          |                        |           |           |          |
*          |                        |           |           v          v
*          |                        |           |  .----------------------------.
*          |                        |           |  | nvvk::RaytracingBuilderKHR |
*          |                        |           |  '--------------+-------------'
*          |                        |           |                 |
*          |                         '----------●----------------'
*          |                                    |
*          |     .----------------.             |
*          |     | VkShaderModule |   vkUpdateDescriptorSets
*          |     '-------+--------'             |
*          |             |                      |
*          |             v                      v
*          |      .------------.    .------------------------------.
*          |      | VkPipeline |    | nvvk::DescriptorSetContainer |
*          |      '------+-----'    '---------+--------------------'
*          v             |                   /
* .-----------------.    |                  /
* | VkCommandBuffer |    |                 /
* '--------+--------'    |                /
*          |             v               /
*          +-----vkCmdBindPipeline      /
*          |                           v
*          +-----vkCmdBindDescriptorSets
*          |
*          +-----vkCmdDispatch
*          |
*          '-----vkCmdPipelineBarrier
*
*************************************************************************************

<!-- Chapter -->

# `pNext`: Going Further

## Rendering Variants

Now that you have a ray tracer, why not try modifying the GLSL code to see what kinds of effects you can create? If anything goes wrong, one can always restore from the code in the `vk_mini_path_tracer` directory.

Here are some examples of different effects that one can make:

### Colors

Instead of setting `result.color = vec3(0.7f)` in `getObjectHitInfo`, we can make the surface color depend on the world normal:

~~~ glsl
  result.color = vec3(0.5) + 0.5 * result.worldNormal;
~~~

That will look like this:

![](images/13-normalColors.png)

Or we can color faces that point in the +x direction red and faces that point in the -x direction green, to start to emulate the Cornell Box:

~~~ glsl
  result.color = vec3(0.8f);

  const float dotX = dot(result.worldNormal, vec3(1.0, 0.0, 0.0));
  if(dotX > 0.99)
  {
    result.color = vec3(0.8, 0.0, 0.0);
  }
  else if(dotX < -0.99)
  {
    result.color = vec3(0.0, 0.8, 0.0);
  }
~~~

![](images/13-cornell.png)

One can see how light picks up color as it bounces from the sky into the box, including the two non-gray walls.

### Glossy Surfaces

We can add the random point on the sphere to the reflected ray to get a glossy material, by replacing the code after `rayOrigin = hitInfo.worldPosition...` with the following:

~~~ glsl
        // For a random diffuse bounce direction, we follow the approach of
        // Ray Tracing in One Weekend, and generate a random point on a sphere
        // of radius 1 centered at the normal. This uses the random_unit_vector
        // function from chapter 8.5:
        const float theta = 6.2831853 * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
        const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;  // Random in [-1, 1]
        const float r     = sqrt(1.0 - u * u);
        rayDirection      = reflect(rayDirection, hitInfo.worldNormal) + 0.02 * vec3(r * cos(theta), r * sin(theta), u);
        // Then normalize the ray direction:
        rayDirection = normalize(rayDirection);
~~~

This will look like this:

![](images/13-glossy.png)

!!! Warning
    This will sometimes generate rays that go below the surface.

Or replace the code with the following to use mirror reflection 20% of the time, and use Lambertian reflection 80% of the time:

~~~ glsl
        if(stepAndOutputRNGFloat(rngState) < 0.2)
        {
          rayDirection = reflect(rayDirection, hitInfo.worldNormal);
        }
        else
        {
          // For a random diffuse bounce direction, we follow the approach of
          // Ray Tracing in One Weekend, and generate a random point on a sphere
          // of radius 1 centered at the normal. This uses the random_unit_vector
          // function from chapter 8.5:
          const float theta = 6.2831853 * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
          const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;  // Random in [-1, 1]
          const float r     = sqrt(1.0 - u * u);
          rayDirection      = hitInfo.worldNormal + vec3(r * cos(theta), r * sin(theta), u);
          // Then normalize the ray direction:
          rayDirection = normalize(rayDirection);
        }
~~~

![](images/13-mirrorAndDiffuse.png)

### Transparency

We can apply color absorption, but make the ray pass through the surface, by changing `origin` and leaving `rayDirection` unchanged, by changing the code after `hitInfo.worldNormal = faceforward...` to:

~~~ glsl
        // Note the minus sign here!
        rayOrigin = hitInfo.worldPosition - 0.0001 * hitInfo.worldNormal;
~~~

![](images/13-transparency.png)

This may not look interesting (because all surfaces have the same color), but this gives a way to do order-independent transparency quickly without additional data structures.

We can make it so that the ray uses diffuse reflection 50% of the time, but goes through the surface the other 50% of the time, by replacing the code as follows:

~~~ glsl
        if(stepAndOutputRNGFloat(rngState) < 0.5)
        {
          // Start a new ray at the hit position, but offset it slightly along
          // the normal against rayDirection:
          rayOrigin = hitInfo.worldPosition + 0.0001 * hitInfo.worldNormal;

          // For a random diffuse bounce direction, we follow the approach of
          // Ray Tracing in One Weekend, and generate a random point on a sphere
          // of radius 1 centered at the normal. This uses the random_unit_vector
          // function from chapter 8.5:
          const float theta = 6.2831853 * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
          const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;  // Random in [-1, 1]
          const float r     = sqrt(1.0 - u * u);
          rayDirection      = hitInfo.worldNormal + vec3(r * cos(theta), r * sin(theta), u);
          // Then normalize the ray direction:
          rayDirection = normalize(rayDirection);
        }
        else
        {
          // Note the minus sign here!
          rayOrigin = hitInfo.worldPosition - 0.0001 * hitInfo.worldNormal;
        }
~~~

This is a technique for stochastic transparency with global illumination.

![](images/13-transparencyDiffuse.png)

### Cutouts

Here's how to make rays pass through slices of the model, making them invisible. Notice that lighting still works -- light can now enter the box from more directions! To do this, replace the code after `HitInfo hitInfo = getObjectHitInfo(rayQuery)` with

~~~ glsl
        const vec3 doubleNormal = faceforward(hitInfo.worldNormal, rayDirection, hitInfo.worldNormal);

        if(mod(dot(hitInfo.worldPosition, vec3(1, 1, 1)), 0.5) < 0.25)
        {
          // Pass through the surface (note the sign)
          rayOrigin = hitInfo.worldPosition - 0.0001 * doubleNormal;
        }
        else
        {
          // Apply color absorption
          accumulatedRayColor *= hitInfo.color;

          // Start a new ray at the hit position, but offset it slightly along
          // the normal against rayDirection:
          rayOrigin = hitInfo.worldPosition + 0.0001 * doubleNormal;

          // For a random diffuse bounce direction, we follow the approach of
          // Ray Tracing in One Weekend, and generate a random point on a sphere
          // of radius 1 centered at the normal. This uses the random_unit_vector
          // function from chapter 8.5:
          const float theta = 6.2831853 * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
          const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;  // Random in [-1, 1]
          const float r     = sqrt(1.0 - u * u);
          rayDirection      = doubleNormal + vec3(r * cos(theta), r * sin(theta), u);
          // Then normalize the ray direction:
          rayDirection = normalize(rayDirection);
        }
~~~

Note that we've also introduced `doubleNormal`, the double-sided normal, to make Lambertian reflection more plausible when the ray intersects a surface from behind.

![](images/13-slices.png)

## Further Reading

If you're interested in going further with Vulkan ray tracing, check out the [NVIDIA Vulkan Ray Tracing Tutorials](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR) by Martin-Karl Lefrançois, Pascal Gautron, Nia Bickford, and David Akeley. These show how to create things like:
* An interactive ray tracer, so that you can move around and change the scene in real-time;
* Ray tracing pipelines, which allow you to efficiently render scenes with thousands of materials -- and even call shaders from other shaders;
* Using instancing to render large numbers of objects
* Procedural intersection shaders (define objects mathematically, without using meshes!);
* Animating ray traced scenes in real-time;
* and much more!

On the topic of path tracing itself, Peter Shirley has written three introduction-to-ray-tracing books, all of which are free online [here](https://raytracing.github.io/). If you've finished this tutorial, [Chapter 10 of Ray Tracing in One Weekend](https://raytracing.github.io/books/RayTracingInOneWeekend.html#dielectrics) is a good place to continue. These books implement a CPU-based renderer, but a Vulkan implementation is available [here](https://github.com/GPSnoopy/RayTracingInVulkan), and the underlying techniques don't depend as much on the API. (In addition to speed, using Vulkan ray tracing also means that you can skip chapter 3 of *Ray Tracing: The Next Week*.)

If you're interested in finding out how to implement other techniques in Vulkan, the [NVIDIA DesignWorks Samples page](https://github.com/nvpro-samples/build_all) (of which this is a part) includes many other Vulkan and OpenGL samples, from [using the OptiX 7 AI denoiser](https://github.com/nvpro-samples/vk_denoise) to [device-generated commands](https://github.com/nvpro-samples/vk_device_generated_cmds) to [OpenGL-Vulkan interop](https://github.com/nvpro-samples/gl_vk_raytrace_interop).

## Extra Chapters

We've also included some extra chapters that go deeper into Vulkan features, and show how to implement some speed and quality improvements in the `vk_mini_path_tracer`, such as:
- A Gaussian filter for antialiasing;
- Measuring performance;
- Compacting acceleration structures;
- Using push constants;
- Using Vulkan images instead of Vulkan buffers to store colors;
- And rendering thousands of samples without running into Timeout Detection and Recovery.

This program is ultimately able to render more than 2,000 samples per pixel for an 800 x 600 image in less than 2 seconds on an RTX 2070.

To read the extras, go here:

▶ **[vk_mini_path_tracer: Extra Chapters](extras.html)**

</script>

<!-- Markdeep: -->
<link rel="stylesheet" href="vkrt_tutorial.css?">
<script> window.markdeepOptions = { tocStyle: "medium" };</script>
<script src="markdeep.min.js" charset="utf-8"></script>
<script src="https://developer.download.nvidia.com/ProGraphics/nvpro-samples/scripts/markdeep.min.js" charset="utf-8"></script>
<script>
    window.alreadyProcessedMarkdeep || (document.body.style.visibility = "visible")
</script>
